2023-10-01T19:24:19.851 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-01T19:24:19.851 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414 --name adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi --description orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} --verbose --owner scheduled_adking@teuthology -- /home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414/orig.config.yaml 2023-10-01T19:24:19.896 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-01T19:24:19.976 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414 branch: wip-adk-testing-2023-10-01-1310 description: orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} email: adking@redhat.com first_in_suite: false job_id: '7408414' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-adk-testing-2023-10-01-1310 ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: true mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 37d71fd39f09a164b3518d8aef9e20133197e5f9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: 37d71fd39f09a164b3518d8aef9e20133197e5f9 workunit: branch: wip-adk-testing-2023-10-01-1310 sha1: 37d71fd39f09a164b3518d8aef9e20133197e5f9 owner: scheduled_adking@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b seed: 6286 sha1: 37d71fd39f09a164b3518d8aef9e20133197e5f9 sleep_before_teardown: 0 subset: 1/14 suite: orch:cephadm suite_branch: wip-adk-testing-2023-10-01-1310 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_37d71fd39f09a164b3518d8aef9e20133197e5f9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 37d71fd39f09a164b3518d8aef9e20133197e5f9 targets: smithi057.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK8YYlq2Rp06VHBEX27K/8AkZc8R3eU+RtuZM71chaqD+0nnh/8jTrdaEObMg5q0X09JtzTT68xaayDSm/pyU3A= smithi087.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEzI4F+XBj70CitNOaqAwYigOp8SsukyIckXGTxr4S3m4S3WXm5I+CNA4etqkLkOk6UqNbV5z3YR8LD+RguoOH0= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - cephadm: cephadm_branch: v17.2.0 cephadm_git_url: https://github.com/ceph/ceph image: quay.io/ceph/ceph:v17.2.0 - cephadm.shell: env: - sha1 mon.a: - radosgw-admin realm create --rgw-realm=r --default - radosgw-admin zonegroup create --rgw-zonegroup=default --master --default - radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=z --master --default - radosgw-admin period update --rgw-realm=r --commit - ceph orch apply rgw foo --realm r --zone z --placement=2 --port=8000 - ceph osd pool create foo - rbd pool init foo - ceph orch apply iscsi foo u p - sleep 180 - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch ps - ceph versions - ceph -s - ceph orch ls - ceph orch daemon redeploy "mgr.$(ceph mgr dump -f json | jq .standbys | jq .[] | jq -r .name)" --image quay.ceph.io/ceph-ci/ceph:$sha1 - ceph orch ps --refresh - sleep 180 - ceph orch ps - ceph versions - ceph -s - ceph health detail - ceph versions | jq -e '.mgr | length == 2' - ceph mgr fail - sleep 180 - ceph orch daemon redeploy "mgr.$(ceph mgr dump -f json | jq .standbys | jq .[] | jq -r .name)" --image quay.ceph.io/ceph-ci/ceph:$sha1 - ceph orch ps --refresh - sleep 180 - ceph orch ps - ceph versions - ceph health detail - ceph -s - ceph mgr fail - sleep 180 - ceph orch ps - ceph versions - ceph -s - ceph health detail - ceph versions | jq -e '.mgr | length == 1' - ceph mgr fail - sleep 180 - ceph orch ps - ceph orch ls - ceph versions - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch upgrade status - ceph health detail - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mon --hosts $(ceph orch ps | grep mgr.x | awk '{print $2}') - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph orch ps - ceph versions | jq -e '.mon | length == 2' - ceph orch upgrade status - ceph health detail - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mon --hosts $(ceph orch ps | grep mgr.y | awk '{print $2}') - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph orch ps - ceph versions | jq -e '.mon | length == 1' - ceph versions | jq -e '.mon | keys' | grep $sha1 - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 5' - ceph orch upgrade status - ceph health detail - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types osd --limit 2 - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph orch ps - ceph versions | jq -e '.osd | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 7' - ceph orch upgrade status - ceph health detail - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types crash,osd --limit 1 - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph orch ps - ceph versions | jq -e '.osd | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 8' - ceph orch upgrade status - ceph health detail - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types crash,osd - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph orch ps - ceph versions | jq -e '.osd | length == 1' - ceph versions | jq -e '.osd | keys' | grep $sha1 - ceph orch upgrade status - ceph health detail - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --services rgw.foo - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph orch ps - ceph versions | jq -e '.rgw | length == 1' - ceph versions | jq -e '.rgw | keys' | grep $sha1 - ceph orch upgrade status - ceph health detail - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 mon.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph orch upgrade status - ceph health detail - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 - ceph orch ls | grep '^osd ' - cephadm.shell: mon.a: - ceph orch upgrade ls - ceph orch upgrade ls --image quay.io/ceph/ceph --show-all-versions | grep 16.2.0 - ceph orch upgrade ls --image quay.io/ceph/ceph --tags | grep v16.2.2 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-01_19:14:56 tube: smithi user: adking verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-01T19:24:19.977 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_37d71fd39f09a164b3518d8aef9e20133197e5f9/qa; will attempt to use it 2023-10-01T19:24:19.978 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_37d71fd39f09a164b3518d8aef9e20133197e5f9/qa/tasks 2023-10-01T19:24:19.978 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-01T19:24:19.979 INFO:teuthology.task.internal:Checking packages... 2023-10-01T19:24:20.003 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '37d71fd39f09a164b3518d8aef9e20133197e5f9' 2023-10-01T19:24:20.003 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-01T19:24:20.003 INFO:teuthology.packaging:ref: None 2023-10-01T19:24:20.003 INFO:teuthology.packaging:tag: None 2023-10-01T19:24:20.003 INFO:teuthology.packaging:branch: wip-adk-testing-2023-10-01-1310 2023-10-01T19:24:20.003 INFO:teuthology.packaging:sha1: 37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T19:24:20.004 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-adk-testing-2023-10-01-1310 2023-10-01T19:24:20.229 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6505.g37d71fd3 2023-10-01T19:24:20.229 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-01T19:24:20.236 INFO:teuthology.task.internal:no buildpackages task found 2023-10-01T19:24:20.236 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-01T19:24:20.243 INFO:teuthology.task.internal:Saving configuration 2023-10-01T19:24:20.262 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-01T19:24:20.268 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-01T19:24:20.294 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-01 19:16:11.702963', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK8YYlq2Rp06VHBEX27K/8AkZc8R3eU+RtuZM71chaqD+0nnh/8jTrdaEObMg5q0X09JtzTT68xaayDSm/pyU3A='} 2023-10-01T19:24:20.315 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-01 19:16:11.704601', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEzI4F+XBj70CitNOaqAwYigOp8SsukyIckXGTxr4S3m4S3WXm5I+CNA4etqkLkOk6UqNbV5z3YR8LD+RguoOH0='} 2023-10-01T19:24:20.315 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-01T19:24:20.323 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2023-10-01T19:24:20.323 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2023-10-01T19:24:20.323 INFO:teuthology.run_tasks:Running task console_log... 2023-10-01T19:24:20.396 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faa5f2fdb80>, signals=[15]) 2023-10-01T19:24:20.396 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-01T19:24:20.402 INFO:teuthology.task.internal:Opening connections... 2023-10-01T19:24:20.402 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi057.front.sepia.ceph.com 2023-10-01T19:24:20.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-01T19:24:20.473 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi087.front.sepia.ceph.com 2023-10-01T19:24:20.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-01T19:24:20.542 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-01T19:24:20.548 DEBUG:teuthology.orchestra.run.smithi057:> uname -m 2023-10-01T19:24:20.571 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2023-10-01T19:24:20.571 DEBUG:teuthology.orchestra.run.smithi057:> cat /etc/os-release 2023-10-01T19:24:20.633 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="CentOS Stream" 2023-10-01T19:24:20.633 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="8" 2023-10-01T19:24:20.633 INFO:teuthology.orchestra.run.smithi057.stdout:ID="centos" 2023-10-01T19:24:20.633 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE="rhel fedora" 2023-10-01T19:24:20.633 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="8" 2023-10-01T19:24:20.633 INFO:teuthology.orchestra.run.smithi057.stdout:PLATFORM_ID="platform:el8" 2023-10-01T19:24:20.633 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-01T19:24:20.633 INFO:teuthology.orchestra.run.smithi057.stdout:ANSI_COLOR="0;31" 2023-10-01T19:24:20.633 INFO:teuthology.orchestra.run.smithi057.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-01T19:24:20.633 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://centos.org/" 2023-10-01T19:24:20.633 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-01T19:24:20.634 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-01T19:24:20.634 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-01T19:24:20.634 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2023-10-01T19:24:20.699 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2023-10-01T19:24:20.722 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2023-10-01T19:24:20.722 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2023-10-01T19:24:20.783 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream" 2023-10-01T19:24:20.783 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8" 2023-10-01T19:24:20.783 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos" 2023-10-01T19:24:20.783 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora" 2023-10-01T19:24:20.783 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8" 2023-10-01T19:24:20.783 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2023-10-01T19:24:20.783 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-01T19:24:20.783 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2023-10-01T19:24:20.784 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-01T19:24:20.784 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/" 2023-10-01T19:24:20.784 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-01T19:24:20.784 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-01T19:24:20.784 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-01T19:24:20.784 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2023-10-01T19:24:20.807 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-01T19:24:20.816 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-01T19:24:20.821 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-01T19:24:20.822 DEBUG:teuthology.orchestra.run.smithi057:> test '!' -e /home/ubuntu/cephtest 2023-10-01T19:24:20.824 DEBUG:teuthology.orchestra.run.smithi087:> test '!' -e /home/ubuntu/cephtest 2023-10-01T19:24:20.844 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-01T19:24:20.850 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-01T19:24:20.850 DEBUG:teuthology.orchestra.run.smithi057:> test -z $(ls -A /var/lib/ceph) 2023-10-01T19:24:20.887 DEBUG:teuthology.orchestra.run.smithi087:> test -z $(ls -A /var/lib/ceph) 2023-10-01T19:24:20.917 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-01T19:24:20.972 INFO:teuthology.run_tasks:Running task kernel... 2023-10-01T19:24:20.989 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-01T19:24:20.989 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-01T19:24:20.990 DEBUG:teuthology.orchestra.run.smithi057:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-01T19:24:20.990 DEBUG:teuthology.orchestra.run.smithi087:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-01T19:24:21.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-01T19:24:21.011 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2023-10-01T19:24:21.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-01T19:24:21.012 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2023-10-01T19:24:21.070 INFO:teuthology.orchestra.run.smithi087.stdout:4.18.0-489.el8.x86_64 2023-10-01T19:24:21.070 INFO:teuthology.task.kernel:Running kernel on smithi087: 4.18.0-489.el8.x86_64 2023-10-01T19:24:21.070 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum install -y kernel 2023-10-01T19:24:21.074 INFO:teuthology.orchestra.run.smithi057.stdout:4.18.0-489.el8.x86_64 2023-10-01T19:24:21.075 INFO:teuthology.task.kernel:Running kernel on smithi057: 4.18.0-489.el8.x86_64 2023-10-01T19:24:21.075 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum install -y kernel 2023-10-01T19:24:24.864 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - AppStream 26 MB/s | 33 MB 00:01 2023-10-01T19:24:24.949 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 8 - AppStream 26 MB/s | 33 MB 00:01 2023-10-01T19:24:33.530 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - BaseOS 22 MB/s | 48 MB 00:02 2023-10-01T19:24:34.566 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 8 - BaseOS 13 MB/s | 48 MB 00:03 2023-10-01T19:24:41.213 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - Extras 50 kB/s | 18 kB 00:00 2023-10-01T19:24:41.799 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 8 - Extras 48 kB/s | 18 kB 00:00 2023-10-01T19:24:52.255 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - Extras common packages 655 B/s | 6.9 kB 00:10 2023-10-01T19:24:52.564 INFO:teuthology.orchestra.run.smithi057.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-10-01T19:24:54.140 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2023-10-01T19:24:55.499 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 8 - Extras common packages 526 B/s | 6.9 kB 00:13 2023-10-01T19:24:55.892 INFO:teuthology.orchestra.run.smithi087.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-10-01T19:24:57.601 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-10-01T19:24:58.223 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 307 kB/s | 24 kB 00:00 2023-10-01T19:25:01.368 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 444 kB/s | 24 kB 00:00 2023-10-01T19:25:01.939 INFO:teuthology.orchestra.run.smithi057.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-10-01T19:25:01.939 INFO:teuthology.orchestra.run.smithi057.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-10-01T19:25:02.210 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-10-01T19:25:02.210 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-10-01T19:25:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2023-10-01T19:25:02.211 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-10-01T19:25:02.211 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2023-10-01T19:25:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-10-01T19:25:02.211 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2023-10-01T19:25:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-10-01T19:25:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-10-01T19:25:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:25:02.211 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2023-10-01T19:25:02.211 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-10-01T19:25:02.211 INFO:teuthology.orchestra.run.smithi057.stdout:Install 3 Packages 2023-10-01T19:25:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:25:02.212 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 88 M 2023-10-01T19:25:02.212 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 96 M 2023-10-01T19:25:02.212 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2023-10-01T19:25:03.024 INFO:teuthology.orchestra.run.smithi057.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 13 MB/s | 10 MB 00:00 2023-10-01T19:25:03.783 INFO:teuthology.orchestra.run.smithi057.stdout:(2/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 23 MB/s | 35 MB 00:01 2023-10-01T19:25:04.283 INFO:teuthology.orchestra.run.smithi057.stdout:(3/3): kernel-core-4.18.0-514.el8.x86_64.rpm 21 MB/s | 43 MB 00:02 2023-10-01T19:25:04.284 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2023-10-01T19:25:04.284 INFO:teuthology.orchestra.run.smithi057.stdout:Total 42 MB/s | 88 MB 00:02 2023-10-01T19:25:04.909 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-10-01T19:25:04.909 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-10-01T19:25:04.980 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2023-10-01T19:25:05.154 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-10-01T19:25:05.155 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-10-01T19:25:05.155 INFO:teuthology.orchestra.run.smithi087.stdout: Package Architecture Version Repository Size 2023-10-01T19:25:05.155 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-10-01T19:25:05.155 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-10-01T19:25:05.155 INFO:teuthology.orchestra.run.smithi087.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-10-01T19:25:05.155 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2023-10-01T19:25:05.155 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-10-01T19:25:05.155 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-10-01T19:25:05.155 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:25:05.155 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-10-01T19:25:05.156 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-10-01T19:25:05.156 INFO:teuthology.orchestra.run.smithi087.stdout:Install 3 Packages 2023-10-01T19:25:05.156 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:25:05.156 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 88 M 2023-10-01T19:25:05.156 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 96 M 2023-10-01T19:25:05.157 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-10-01T19:25:05.423 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2023-10-01T19:25:05.423 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2023-10-01T19:25:05.911 INFO:teuthology.orchestra.run.smithi087.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 14 MB/s | 10 MB 00:00 2023-10-01T19:25:06.714 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2023-10-01T19:25:06.803 INFO:teuthology.orchestra.run.smithi087.stdout:(2/3): kernel-core-4.18.0-514.el8.x86_64.rpm 26 MB/s | 43 MB 00:01 2023-10-01T19:25:06.826 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2023-10-01T19:25:07.350 INFO:teuthology.orchestra.run.smithi087.stdout:(3/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 16 MB/s | 35 MB 00:02 2023-10-01T19:25:07.351 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-10-01T19:25:07.351 INFO:teuthology.orchestra.run.smithi087.stdout:Total 40 MB/s | 88 MB 00:02 2023-10-01T19:25:08.026 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-10-01T19:25:08.437 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-10-01T19:25:08.437 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-10-01T19:25:09.657 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-10-01T19:25:09.682 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-10-01T19:25:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2023-10-01T19:25:10.522 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-01T19:25:12.433 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-10-01T19:25:12.988 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-01T19:25:14.246 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-01T19:25:14.638 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-01T19:25:16.596 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-01T19:25:16.996 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-01T19:25:18.269 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-01T19:25:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-10-01T19:25:20.369 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-01T19:25:20.661 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-10-01T19:25:41.740 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-10-01T19:25:41.764 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-01T19:25:42.224 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-10-01T19:25:42.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-10-01T19:25:42.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-10-01T19:25:42.500 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-10-01T19:25:42.524 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-01T19:25:42.853 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-10-01T19:25:42.853 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-10-01T19:25:42.853 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-10-01T19:25:43.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-01T19:25:43.229 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:25:43.229 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2023-10-01T19:25:43.229 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-10-01T19:25:43.229 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-10-01T19:25:43.229 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:25:43.229 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-10-01T19:25:43.394 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q kernel | sort -rV | head -n 1 2023-10-01T19:25:43.592 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-01T19:25:43.592 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:25:43.592 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-10-01T19:25:43.592 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-10-01T19:25:43.592 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-10-01T19:25:43.592 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:25:43.592 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-10-01T19:25:43.622 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-01T19:25:43.622 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-01T19:25:43.622 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-10-01T19:25:43.623 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-01T19:25:43.623 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-01T19:25:43.623 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-01T19:25:43.623 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi057.front.sepia.ceph.com, path=None, version=distro) 2023-10-01T19:25:43.623 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q kernel | sort -rV | head -n 1 2023-10-01T19:25:43.841 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-01T19:25:43.841 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-01T19:25:43.841 DEBUG:teuthology.orchestra.run.smithi057:> sudo rpm -qi grub2-tools 2023-10-01T19:25:43.877 INFO:teuthology.orchestra.run.smithi057.stdout:Name : grub2-tools 2023-10-01T19:25:43.877 INFO:teuthology.orchestra.run.smithi057.stdout:Epoch : 1 2023-10-01T19:25:43.877 INFO:teuthology.orchestra.run.smithi057.stdout:Version : 2.02 2023-10-01T19:25:43.878 INFO:teuthology.orchestra.run.smithi057.stdout:Release : 129.el8 2023-10-01T19:25:43.878 INFO:teuthology.orchestra.run.smithi057.stdout:Architecture: x86_64 2023-10-01T19:25:43.878 INFO:teuthology.orchestra.run.smithi057.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-01T19:25:43.878 INFO:teuthology.orchestra.run.smithi057.stdout:Group : System Environment/Base 2023-10-01T19:25:43.878 INFO:teuthology.orchestra.run.smithi057.stdout:Size : 9265586 2023-10-01T19:25:43.878 INFO:teuthology.orchestra.run.smithi057.stdout:License : GPLv3+ 2023-10-01T19:25:43.878 INFO:teuthology.orchestra.run.smithi057.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-01T19:25:43.878 INFO:teuthology.orchestra.run.smithi057.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-01T19:25:43.878 INFO:teuthology.orchestra.run.smithi057.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-01T19:25:43.878 INFO:teuthology.orchestra.run.smithi057.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-01T19:25:43.878 INFO:teuthology.orchestra.run.smithi057.stdout:Relocations : (not relocatable) 2023-10-01T19:25:43.878 INFO:teuthology.orchestra.run.smithi057.stdout:Packager : CentOS BuildSystem 2023-10-01T19:25:43.879 INFO:teuthology.orchestra.run.smithi057.stdout:Vendor : CentOS 2023-10-01T19:25:43.879 INFO:teuthology.orchestra.run.smithi057.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-01T19:25:43.879 INFO:teuthology.orchestra.run.smithi057.stdout:Summary : Support tools for GRUB. 2023-10-01T19:25:43.879 INFO:teuthology.orchestra.run.smithi057.stdout:Description : 2023-10-01T19:25:43.879 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:25:43.879 INFO:teuthology.orchestra.run.smithi057.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-01T19:25:43.879 INFO:teuthology.orchestra.run.smithi057.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-01T19:25:43.879 INFO:teuthology.orchestra.run.smithi057.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-01T19:25:43.879 INFO:teuthology.orchestra.run.smithi057.stdout:hardware devices. 2023-10-01T19:25:43.879 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:25:43.879 INFO:teuthology.orchestra.run.smithi057.stdout:This subpackage provides tools for support of all platforms. 2023-10-01T19:25:43.879 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-01T19:25:43.880 INFO:teuthology.task.kernel:Updating grub on smithi057 to boot 4.18.0-514.el8.x86_64 2023-10-01T19:25:43.880 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-01T19:25:43.926 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel | sort -rV | head -n 1 2023-10-01T19:25:44.142 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-01T19:25:44.142 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-01T19:25:44.142 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-10-01T19:25:44.142 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-01T19:25:44.142 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-01T19:25:44.142 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-01T19:25:44.142 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi087.front.sepia.ceph.com, path=None, version=distro) 2023-10-01T19:25:44.142 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel | sort -rV | head -n 1 2023-10-01T19:25:44.350 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-01T19:25:44.350 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-01T19:25:44.351 DEBUG:teuthology.orchestra.run.smithi087:> sudo rpm -qi grub2-tools 2023-10-01T19:25:44.384 INFO:teuthology.orchestra.run.smithi087.stdout:Name : grub2-tools 2023-10-01T19:25:44.384 INFO:teuthology.orchestra.run.smithi087.stdout:Epoch : 1 2023-10-01T19:25:44.384 INFO:teuthology.orchestra.run.smithi087.stdout:Version : 2.02 2023-10-01T19:25:44.384 INFO:teuthology.orchestra.run.smithi087.stdout:Release : 129.el8 2023-10-01T19:25:44.384 INFO:teuthology.orchestra.run.smithi087.stdout:Architecture: x86_64 2023-10-01T19:25:44.384 INFO:teuthology.orchestra.run.smithi087.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-01T19:25:44.385 INFO:teuthology.orchestra.run.smithi087.stdout:Group : System Environment/Base 2023-10-01T19:25:44.385 INFO:teuthology.orchestra.run.smithi087.stdout:Size : 9265586 2023-10-01T19:25:44.385 INFO:teuthology.orchestra.run.smithi087.stdout:License : GPLv3+ 2023-10-01T19:25:44.385 INFO:teuthology.orchestra.run.smithi087.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-01T19:25:44.385 INFO:teuthology.orchestra.run.smithi087.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-01T19:25:44.385 INFO:teuthology.orchestra.run.smithi087.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-01T19:25:44.385 INFO:teuthology.orchestra.run.smithi087.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-01T19:25:44.385 INFO:teuthology.orchestra.run.smithi087.stdout:Relocations : (not relocatable) 2023-10-01T19:25:44.385 INFO:teuthology.orchestra.run.smithi087.stdout:Packager : CentOS BuildSystem 2023-10-01T19:25:44.385 INFO:teuthology.orchestra.run.smithi087.stdout:Vendor : CentOS 2023-10-01T19:25:44.386 INFO:teuthology.orchestra.run.smithi087.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-01T19:25:44.386 INFO:teuthology.orchestra.run.smithi087.stdout:Summary : Support tools for GRUB. 2023-10-01T19:25:44.386 INFO:teuthology.orchestra.run.smithi087.stdout:Description : 2023-10-01T19:25:44.386 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:25:44.386 INFO:teuthology.orchestra.run.smithi087.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-01T19:25:44.386 INFO:teuthology.orchestra.run.smithi087.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-01T19:25:44.386 INFO:teuthology.orchestra.run.smithi087.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-01T19:25:44.386 INFO:teuthology.orchestra.run.smithi087.stdout:hardware devices. 2023-10-01T19:25:44.386 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:25:44.386 INFO:teuthology.orchestra.run.smithi087.stdout:This subpackage provides tools for support of all platforms. 2023-10-01T19:25:44.387 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-01T19:25:44.387 INFO:teuthology.task.kernel:Updating grub on smithi087 to boot 4.18.0-514.el8.x86_64 2023-10-01T19:25:44.388 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-01T19:25:45.087 INFO:teuthology.orchestra.run.smithi057.stderr:Generating grub configuration file ... 2023-10-01T19:25:45.755 INFO:teuthology.orchestra.run.smithi087.stderr:Generating grub configuration file ... 2023-10-01T19:25:46.965 INFO:teuthology.orchestra.run.smithi057.stderr:done 2023-10-01T19:25:46.970 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2023-10-01T19:25:47.006 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.vOefSqKjXH 2023-10-01T19:25:47.006 DEBUG:teuthology.orchestra.run.smithi057:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.vOefSqKjXH 2023-10-01T19:25:47.080 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.vOefSqKjXH 2023-10-01T19:25:47.226 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.vOefSqKjXH is 6KB 2023-10-01T19:25:47.274 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.vOefSqKjXH 2023-10-01T19:25:47.298 DEBUG:teuthology.orchestra.run.smithi057:> sudo /bin/ls /boot/loader/entries || true 2023-10-01T19:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-01T19:25:47.359 INFO:teuthology.orchestra.run.smithi057.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-01T19:25:47.359 INFO:teuthology.orchestra.run.smithi057.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-01T19:25:47.359 INFO:teuthology.orchestra.run.smithi057.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-10-01T19:25:47.360 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-10-01T19:25:47.558 DEBUG:teuthology.orchestra.run.smithi057:> sudo shutdown -r now 2023-10-01T19:25:48.057 INFO:teuthology.orchestra.run.smithi087.stderr:done 2023-10-01T19:25:48.058 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2023-10-01T19:25:48.087 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.spfvboJ1BL 2023-10-01T19:25:48.088 DEBUG:teuthology.orchestra.run.smithi087:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.spfvboJ1BL 2023-10-01T19:25:48.169 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.spfvboJ1BL 2023-10-01T19:25:48.316 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.spfvboJ1BL is 6KB 2023-10-01T19:25:48.364 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.spfvboJ1BL 2023-10-01T19:25:48.381 DEBUG:teuthology.orchestra.run.smithi087:> sudo /bin/ls /boot/loader/entries || true 2023-10-01T19:25:48.455 INFO:teuthology.orchestra.run.smithi087.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-01T19:25:48.455 INFO:teuthology.orchestra.run.smithi087.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-01T19:25:48.455 INFO:teuthology.orchestra.run.smithi087.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-01T19:25:48.455 INFO:teuthology.orchestra.run.smithi087.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-10-01T19:25:48.456 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-10-01T19:25:48.623 DEBUG:teuthology.orchestra.run.smithi087:> sudo shutdown -r now 2023-10-01T19:26:17.586 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-01T19:26:17.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-01T19:26:17.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-01T19:26:18.627 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-01T19:26:18.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-10-01T19:26:18.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-01T19:26:52.282 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2023-10-01T19:27:01.290 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-01T19:27:01.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-01T19:27:18.646 DEBUG:teuthology.orchestra.remote:timed out 2023-10-01T19:27:27.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-10-01T19:27:27.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-01T19:27:28.105 DEBUG:teuthology.orchestra.run.smithi087:> true 2023-10-01T19:27:28.422 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-10-01T19:27:28.422 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-514.el8.x86_64"... 2023-10-01T19:27:28.423 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2023-10-01T19:27:28.440 INFO:teuthology.orchestra.run.smithi087.stdout:4.18.0-514.el8.x86_64 2023-10-01T19:27:28.441 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-10-01T19:27:28.441 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-01T19:27:28.441 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-01T19:27:29.442 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-01T19:27:29.442 DEBUG:teuthology.orchestra.run.smithi087:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-01T19:27:29.503 INFO:teuthology.orchestra.run.smithi087.stdout:ttyS1 2023-10-01T19:27:29.516 DEBUG:teuthology.parallel:result is None 2023-10-01T19:28:01.293 DEBUG:teuthology.orchestra.remote:timed out 2023-10-01T19:28:13.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-01T19:28:13.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-01T19:28:13.689 DEBUG:teuthology.orchestra.run.smithi057:> true 2023-10-01T19:28:14.041 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-01T19:28:14.041 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-514.el8.x86_64"... 2023-10-01T19:28:14.042 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2023-10-01T19:28:14.065 INFO:teuthology.orchestra.run.smithi057.stdout:4.18.0-514.el8.x86_64 2023-10-01T19:28:14.066 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-10-01T19:28:14.066 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-01T19:28:14.066 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-01T19:28:15.067 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-01T19:28:15.067 DEBUG:teuthology.orchestra.run.smithi057:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-01T19:28:15.176 INFO:teuthology.orchestra.run.smithi057.stdout:ttyS1 2023-10-01T19:28:15.189 DEBUG:teuthology.parallel:result is None 2023-10-01T19:28:15.190 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-01T19:28:15.197 INFO:teuthology.task.internal:Creating test directory... 2023-10-01T19:28:15.197 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-01T19:28:15.200 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-01T19:28:15.226 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-01T19:28:15.235 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-01T19:28:15.245 INFO:teuthology.task.internal:Creating archive directory... 2023-10-01T19:28:15.245 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-01T19:28:15.263 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-01T19:28:15.310 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-01T19:28:15.318 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-01T19:28:15.319 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-01T19:28:15.336 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-01T19:28:15.377 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-01T19:28:15.388 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-01T19:28:15.397 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-01T19:28:15.405 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-01T19:28:15.407 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-01T19:28:15.415 INFO:teuthology.task.internal:Configuring sudo... 2023-10-01T19:28:15.416 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-01T19:28:15.433 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-01T19:28:15.472 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-01T19:28:15.486 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-01T19:28:15.487 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-01T19:28:15.516 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-01T19:28:15.555 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-01T19:28:15.651 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-01T19:28:15.711 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-01T19:28:15.802 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-01T19:28:15.853 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-01T19:28:15.853 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-01T19:28:15.927 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-01T19:28:15.974 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-01T19:28:16.038 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-01T19:28:16.086 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-01T19:28:16.129 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-10-01T19:28:16.130 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-01T19:28:16.199 DEBUG:teuthology.orchestra.run.smithi057:> sudo service rsyslog restart 2023-10-01T19:28:16.203 DEBUG:teuthology.orchestra.run.smithi087:> sudo service rsyslog restart 2023-10-01T19:28:16.280 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-01T19:28:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-01T19:28:16.683 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-01T19:28:16.691 INFO:teuthology.task.internal:Starting timer... 2023-10-01T19:28:16.691 INFO:teuthology.run_tasks:Running task pcp... 2023-10-01T19:28:16.709 INFO:teuthology.run_tasks:Running task selinux... 2023-10-01T19:28:16.724 DEBUG:teuthology.orchestra.run.smithi057:> sudo service auditd rotate 2023-10-01T19:28:16.858 INFO:teuthology.orchestra.run.smithi057.stdout:Rotating logs: 2023-10-01T19:28:16.860 DEBUG:teuthology.orchestra.run.smithi087:> sudo service auditd rotate 2023-10-01T19:28:16.944 INFO:teuthology.orchestra.run.smithi087.stdout:Rotating logs: 2023-10-01T19:28:16.945 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-01T19:28:16.945 DEBUG:teuthology.orchestra.run.smithi057:> /usr/sbin/getenforce 2023-10-01T19:28:17.000 INFO:teuthology.orchestra.run.smithi057.stdout:Permissive 2023-10-01T19:28:17.000 DEBUG:teuthology.orchestra.run.smithi087:> /usr/sbin/getenforce 2023-10-01T19:28:17.037 INFO:teuthology.orchestra.run.smithi087.stdout:Permissive 2023-10-01T19:28:17.037 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi057.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi087.front.sepia.ceph.com': 'permissive'} 2023-10-01T19:28:17.037 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-01T19:28:17.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-01T19:28:17.074 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-01T19:28:17.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-01T19:28:17.106 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-01T19:28:17.106 DEBUG:teuthology.orchestra.run.smithi057:> sudo /usr/sbin/setenforce permissive 2023-10-01T19:28:17.151 DEBUG:teuthology.orchestra.run.smithi087:> sudo /usr/sbin/setenforce permissive 2023-10-01T19:28:17.181 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-01T19:28:17.191 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-01T19:28:17.191 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-01T19:28:17.222 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-01T19:28:17.235 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi057.front.sepia.ceph.com,smithi087.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-01T19:37:27.840 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi057.front.sepia.ceph.com'), Remote(name='ubuntu@smithi087.front.sepia.ceph.com')] 2023-10-01T19:37:27.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-01T19:37:27.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-01T19:37:27.914 DEBUG:teuthology.orchestra.run.smithi057:> true 2023-10-01T19:37:27.992 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-01T19:37:27.993 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-10-01T19:37:27.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-01T19:37:28.063 DEBUG:teuthology.orchestra.run.smithi087:> true 2023-10-01T19:37:28.134 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-10-01T19:37:28.135 INFO:teuthology.run_tasks:Running task clock... 2023-10-01T19:37:28.146 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-01T19:37:28.147 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-01T19:37:28.147 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-01T19:37:28.150 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-01T19:37:28.150 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-01T19:37:28.189 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-01T19:37:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-01T19:37:28.208 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-01T19:37:28.222 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-01T19:37:28.284 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: ntpd: command not found 2023-10-01T19:37:28.297 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: ntpd: command not found 2023-10-01T19:37:28.297 INFO:teuthology.orchestra.run.smithi057.stdout:506 Cannot talk to daemon 2023-10-01T19:37:28.308 INFO:teuthology.orchestra.run.smithi087.stdout:506 Cannot talk to daemon 2023-10-01T19:37:28.314 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-01T19:37:28.322 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-01T19:37:28.330 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-01T19:37:28.336 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-01T19:37:28.397 INFO:teuthology.orchestra.run.smithi057.stderr:bash: ntpq: command not found 2023-10-01T19:37:28.402 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-01T19:37:28.402 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2023-10-01T19:37:28.402 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-01T19:37:28.402 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-01T19:37:28.402 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-01T19:37:28.402 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-01T19:37:28.403 INFO:teuthology.orchestra.run.smithi087.stderr:bash: ntpq: command not found 2023-10-01T19:37:28.406 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-01T19:37:28.406 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2023-10-01T19:37:28.406 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-01T19:37:28.406 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-01T19:37:28.407 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-01T19:37:28.407 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-01T19:37:28.407 INFO:teuthology.run_tasks:Running task pexec... 2023-10-01T19:37:28.417 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-01T19:37:28.417 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi057.front.sepia.ceph.com 2023-10-01T19:37:28.418 DEBUG:teuthology.orchestra.run.smithi057:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-01T19:37:28.418 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi087.front.sepia.ceph.com 2023-10-01T19:37:28.418 DEBUG:teuthology.orchestra.run.smithi087:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-01T19:37:29.331 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:20 ago on Sun 01 Oct 2023 07:35:09 PM UTC. 2023-10-01T19:37:29.459 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:42 ago on Sun 01 Oct 2023 07:35:47 PM UTC. 2023-10-01T19:37:31.346 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-10-01T19:37:31.347 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-10-01T19:37:31.347 INFO:teuthology.orchestra.run.smithi087.stdout: Package Architecture Version Repository Size 2023-10-01T19:37:31.347 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-10-01T19:37:31.347 INFO:teuthology.orchestra.run.smithi087.stdout:Resetting modules: 2023-10-01T19:37:31.347 INFO:teuthology.orchestra.run.smithi087.stdout: container-tools 2023-10-01T19:37:31.348 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:37:31.348 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-10-01T19:37:31.348 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-10-01T19:37:31.348 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:37:31.595 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-10-01T19:37:31.712 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-10-01T19:37:31.713 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-10-01T19:37:31.713 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2023-10-01T19:37:31.713 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-10-01T19:37:31.713 INFO:teuthology.orchestra.run.smithi057.stdout:Resetting modules: 2023-10-01T19:37:31.713 INFO:teuthology.orchestra.run.smithi057.stdout: container-tools 2023-10-01T19:37:31.714 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:37:31.714 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2023-10-01T19:37:31.714 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-10-01T19:37:31.714 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:37:31.982 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-10-01T19:37:32.552 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:23 ago on Sun 01 Oct 2023 07:35:09 PM UTC. 2023-10-01T19:37:33.078 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:45 ago on Sun 01 Oct 2023 07:35:47 PM UTC. 2023-10-01T19:37:34.548 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-10-01T19:37:34.551 INFO:teuthology.orchestra.run.smithi087.stdout:=========================================================================================== 2023-10-01T19:37:34.551 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repo Size 2023-10-01T19:37:34.551 INFO:teuthology.orchestra.run.smithi087.stdout:=========================================================================================== 2023-10-01T19:37:34.551 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrading: 2023-10-01T19:37:34.552 INFO:teuthology.orchestra.run.smithi087.stdout: conmon x86_64 3:2.1.8-1.module_el8+664+4072b3ae appstream 57 k 2023-10-01T19:37:34.552 INFO:teuthology.orchestra.run.smithi087.stdout: container-selinux noarch 2:2.221.0-1.module_el8+653+feef7bfe appstream 69 k 2023-10-01T19:37:34.552 INFO:teuthology.orchestra.run.smithi087.stdout: containernetworking-plugins x86_64 1:1.3.0-4.module_el8+644+cb8a8b65 appstream 21 M 2023-10-01T19:37:34.552 INFO:teuthology.orchestra.run.smithi087.stdout: containers-common x86_64 2:1-54.module_el8+664+4072b3ae appstream 137 k 2023-10-01T19:37:34.552 INFO:teuthology.orchestra.run.smithi087.stdout: criu x86_64 3.18-4.module_el8+454+d7ef4b8d appstream 564 k 2023-10-01T19:37:34.552 INFO:teuthology.orchestra.run.smithi087.stdout: fuse-overlayfs x86_64 1.12-1.module_el8+454+d7ef4b8d appstream 70 k 2023-10-01T19:37:34.552 INFO:teuthology.orchestra.run.smithi087.stdout: libslirp x86_64 4.4.0-1.module_el8+487+8e42a277 appstream 70 k 2023-10-01T19:37:34.552 INFO:teuthology.orchestra.run.smithi087.stdout: runc x86_64 1:1.1.9-1.module_el8+643+8db347f4 appstream 3.1 M 2023-10-01T19:37:34.552 INFO:teuthology.orchestra.run.smithi087.stdout: slirp4netns x86_64 1.2.1-1.module_el8+661+d1afb926 appstream 56 k 2023-10-01T19:37:34.552 INFO:teuthology.orchestra.run.smithi087.stdout:Installing group/module packages: 2023-10-01T19:37:34.552 INFO:teuthology.orchestra.run.smithi087.stdout: aardvark-dns x86_64 2:1.7.0-1.module_el8+487+8e42a277 appstream 1.0 M 2023-10-01T19:37:34.552 INFO:teuthology.orchestra.run.smithi087.stdout: buildah x86_64 1:1.31.3-1.module_el8+664+4072b3ae appstream 8.8 M 2023-10-01T19:37:34.553 INFO:teuthology.orchestra.run.smithi087.stdout: cockpit-podman noarch 75-1.module_el8+664+4072b3ae appstream 739 k 2023-10-01T19:37:34.553 INFO:teuthology.orchestra.run.smithi087.stdout: crun x86_64 1.8.7-1.module_el8+661+d1afb926 appstream 239 k 2023-10-01T19:37:34.553 INFO:teuthology.orchestra.run.smithi087.stdout: netavark x86_64 2:1.7.0-1.module_el8+487+8e42a277 appstream 3.7 M 2023-10-01T19:37:34.553 INFO:teuthology.orchestra.run.smithi087.stdout: python3-podman noarch 4.6.0-1.module_el8+582+a2558ae2 appstream 177 k 2023-10-01T19:37:34.553 INFO:teuthology.orchestra.run.smithi087.stdout: skopeo x86_64 2:1.13.3-1.module_el8+664+4072b3ae appstream 8.1 M 2023-10-01T19:37:34.553 INFO:teuthology.orchestra.run.smithi087.stdout: toolbox x86_64 0.0.99.4-5.module_el8+649+e2deac1d appstream 2.5 M 2023-10-01T19:37:34.553 INFO:teuthology.orchestra.run.smithi087.stdout: udica noarch 0.2.6-20.module_el8+487+8e42a277 appstream 53 k 2023-10-01T19:37:34.553 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2023-10-01T19:37:34.553 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2023-10-01T19:37:34.553 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2023-10-01T19:37:34.553 INFO:teuthology.orchestra.run.smithi087.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2023-10-01T19:37:34.553 INFO:teuthology.orchestra.run.smithi087.stdout:Installing weak dependencies: 2023-10-01T19:37:34.554 INFO:teuthology.orchestra.run.smithi087.stdout: criu-libs x86_64 3.18-4.module_el8+454+d7ef4b8d appstream 39 k 2023-10-01T19:37:34.554 INFO:teuthology.orchestra.run.smithi087.stdout:Installing module profiles: 2023-10-01T19:37:34.554 INFO:teuthology.orchestra.run.smithi087.stdout: container-tools/common 2023-10-01T19:37:34.554 INFO:teuthology.orchestra.run.smithi087.stdout:Enabling module streams: 2023-10-01T19:37:34.554 INFO:teuthology.orchestra.run.smithi087.stdout: container-tools rhel8 2023-10-01T19:37:34.554 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:37:34.554 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-10-01T19:37:34.554 INFO:teuthology.orchestra.run.smithi087.stdout:=========================================================================================== 2023-10-01T19:37:34.555 INFO:teuthology.orchestra.run.smithi087.stdout:Install 13 Packages 2023-10-01T19:37:34.555 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrade 9 Packages 2023-10-01T19:37:34.555 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:37:34.555 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 51 M 2023-10-01T19:37:34.555 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-10-01T19:37:35.008 INFO:teuthology.orchestra.run.smithi087.stdout:(1/22): cockpit-podman-75-1.module_el8+664+4072 1.8 MB/s | 739 kB 00:00 2023-10-01T19:37:35.049 INFO:teuthology.orchestra.run.smithi087.stdout:(2/22): aardvark-dns-1.7.0-1.module_el8+487+8e4 2.2 MB/s | 1.0 MB 00:00 2023-10-01T19:37:35.074 INFO:teuthology.orchestra.run.smithi087.stdout:(3/22): criu-libs-3.18-4.module_el8+454+d7ef4b8 593 kB/s | 39 kB 00:00 2023-10-01T19:37:35.125 INFO:teuthology.orchestra.run.smithi087.stdout:(4/22): crun-1.8.7-1.module_el8+661+d1afb926.x8 3.1 MB/s | 239 kB 00:00 2023-10-01T19:37:35.200 INFO:teuthology.orchestra.run.smithi087.stdout:(5/22): python3-podman-4.6.0-1.module_el8+582+a 2.3 MB/s | 177 kB 00:00 2023-10-01T19:37:35.300 INFO:teuthology.orchestra.run.smithi087.stdout:(6/22): netavark-1.7.0-1.module_el8+487+8e42a27 17 MB/s | 3.7 MB 00:00 2023-10-01T19:37:35.316 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-10-01T19:37:35.319 INFO:teuthology.orchestra.run.smithi057.stdout:=========================================================================================== 2023-10-01T19:37:35.319 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repo Size 2023-10-01T19:37:35.320 INFO:teuthology.orchestra.run.smithi057.stdout:=========================================================================================== 2023-10-01T19:37:35.320 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrading: 2023-10-01T19:37:35.320 INFO:teuthology.orchestra.run.smithi057.stdout: conmon x86_64 3:2.1.8-1.module_el8+664+4072b3ae appstream 57 k 2023-10-01T19:37:35.320 INFO:teuthology.orchestra.run.smithi057.stdout: container-selinux noarch 2:2.221.0-1.module_el8+653+feef7bfe appstream 69 k 2023-10-01T19:37:35.320 INFO:teuthology.orchestra.run.smithi057.stdout: containernetworking-plugins x86_64 1:1.3.0-4.module_el8+644+cb8a8b65 appstream 21 M 2023-10-01T19:37:35.320 INFO:teuthology.orchestra.run.smithi057.stdout: containers-common x86_64 2:1-54.module_el8+664+4072b3ae appstream 137 k 2023-10-01T19:37:35.320 INFO:teuthology.orchestra.run.smithi057.stdout: criu x86_64 3.18-4.module_el8+454+d7ef4b8d appstream 564 k 2023-10-01T19:37:35.320 INFO:teuthology.orchestra.run.smithi057.stdout: fuse-overlayfs x86_64 1.12-1.module_el8+454+d7ef4b8d appstream 70 k 2023-10-01T19:37:35.321 INFO:teuthology.orchestra.run.smithi057.stdout: libslirp x86_64 4.4.0-1.module_el8+487+8e42a277 appstream 70 k 2023-10-01T19:37:35.321 INFO:teuthology.orchestra.run.smithi057.stdout: runc x86_64 1:1.1.9-1.module_el8+643+8db347f4 appstream 3.1 M 2023-10-01T19:37:35.321 INFO:teuthology.orchestra.run.smithi057.stdout: slirp4netns x86_64 1.2.1-1.module_el8+661+d1afb926 appstream 56 k 2023-10-01T19:37:35.321 INFO:teuthology.orchestra.run.smithi057.stdout:Installing group/module packages: 2023-10-01T19:37:35.321 INFO:teuthology.orchestra.run.smithi057.stdout: aardvark-dns x86_64 2:1.7.0-1.module_el8+487+8e42a277 appstream 1.0 M 2023-10-01T19:37:35.321 INFO:teuthology.orchestra.run.smithi057.stdout: buildah x86_64 1:1.31.3-1.module_el8+664+4072b3ae appstream 8.8 M 2023-10-01T19:37:35.321 INFO:teuthology.orchestra.run.smithi057.stdout: cockpit-podman noarch 75-1.module_el8+664+4072b3ae appstream 739 k 2023-10-01T19:37:35.322 INFO:teuthology.orchestra.run.smithi057.stdout: crun x86_64 1.8.7-1.module_el8+661+d1afb926 appstream 239 k 2023-10-01T19:37:35.322 INFO:teuthology.orchestra.run.smithi057.stdout: netavark x86_64 2:1.7.0-1.module_el8+487+8e42a277 appstream 3.7 M 2023-10-01T19:37:35.322 INFO:teuthology.orchestra.run.smithi057.stdout: python3-podman noarch 4.6.0-1.module_el8+582+a2558ae2 appstream 177 k 2023-10-01T19:37:35.322 INFO:teuthology.orchestra.run.smithi057.stdout: skopeo x86_64 2:1.13.3-1.module_el8+664+4072b3ae appstream 8.1 M 2023-10-01T19:37:35.322 INFO:teuthology.orchestra.run.smithi057.stdout: toolbox x86_64 0.0.99.4-5.module_el8+649+e2deac1d appstream 2.5 M 2023-10-01T19:37:35.322 INFO:teuthology.orchestra.run.smithi057.stdout: udica noarch 0.2.6-20.module_el8+487+8e42a277 appstream 53 k 2023-10-01T19:37:35.322 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2023-10-01T19:37:35.322 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2023-10-01T19:37:35.323 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2023-10-01T19:37:35.323 INFO:teuthology.orchestra.run.smithi057.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2023-10-01T19:37:35.323 INFO:teuthology.orchestra.run.smithi057.stdout:Installing weak dependencies: 2023-10-01T19:37:35.323 INFO:teuthology.orchestra.run.smithi057.stdout: criu-libs x86_64 3.18-4.module_el8+454+d7ef4b8d appstream 39 k 2023-10-01T19:37:35.323 INFO:teuthology.orchestra.run.smithi057.stdout:Installing module profiles: 2023-10-01T19:37:35.323 INFO:teuthology.orchestra.run.smithi057.stdout: container-tools/common 2023-10-01T19:37:35.323 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling module streams: 2023-10-01T19:37:35.324 INFO:teuthology.orchestra.run.smithi057.stdout: container-tools rhel8 2023-10-01T19:37:35.324 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:37:35.324 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2023-10-01T19:37:35.324 INFO:teuthology.orchestra.run.smithi057.stdout:=========================================================================================== 2023-10-01T19:37:35.324 INFO:teuthology.orchestra.run.smithi057.stdout:Install 13 Packages 2023-10-01T19:37:35.324 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrade 9 Packages 2023-10-01T19:37:35.324 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:37:35.325 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 51 M 2023-10-01T19:37:35.325 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2023-10-01T19:37:35.325 INFO:teuthology.orchestra.run.smithi087.stdout:(7/22): python3-pytoml-0.1.14-5.git7dea353.el8. 197 kB/s | 25 kB 00:00 2023-10-01T19:37:35.426 INFO:teuthology.orchestra.run.smithi087.stdout:(8/22): buildah-1.31.3-1.module_el8+664+4072b3a 11 MB/s | 8.8 MB 00:00 2023-10-01T19:37:35.451 INFO:teuthology.orchestra.run.smithi087.stdout:(9/22): python3-pyxdg-0.25-16.el8.noarch.rpm 623 kB/s | 94 kB 00:00 2023-10-01T19:37:35.526 INFO:teuthology.orchestra.run.smithi087.stdout:(10/22): toolbox-0.0.99.4-5.module_el8+649+e2de 25 MB/s | 2.5 MB 00:00 2023-10-01T19:37:35.552 INFO:teuthology.orchestra.run.smithi087.stdout:(11/22): udica-0.2.6-20.module_el8+487+8e42a277 524 kB/s | 53 kB 00:00 2023-10-01T19:37:35.593 INFO:teuthology.orchestra.run.smithi087.stdout:(12/22): yajl-2.1.0-12.el8.x86_64.rpm 612 kB/s | 41 kB 00:00 2023-10-01T19:37:35.619 INFO:teuthology.orchestra.run.smithi087.stdout:(13/22): conmon-2.1.8-1.module_el8+664+4072b3ae 858 kB/s | 57 kB 00:00 2023-10-01T19:37:35.661 INFO:teuthology.orchestra.run.smithi087.stdout:(14/22): container-selinux-2.221.0-1.module_el8 1.0 MB/s | 69 kB 00:00 2023-10-01T19:37:35.761 INFO:teuthology.orchestra.run.smithi087.stdout:(15/22): skopeo-1.13.3-1.module_el8+664+4072b3a 19 MB/s | 8.1 MB 00:00 2023-10-01T19:37:35.778 INFO:teuthology.orchestra.run.smithi087.stdout:(16/22): containers-common-1-54.module_el8+664+ 1.1 MB/s | 137 kB 00:00 2023-10-01T19:37:35.853 INFO:teuthology.orchestra.run.smithi087.stdout:(17/22): criu-3.18-4.module_el8+454+d7ef4b8d.x8 6.0 MB/s | 564 kB 00:00 2023-10-01T19:37:35.879 INFO:teuthology.orchestra.run.smithi087.stdout:(18/22): fuse-overlayfs-1.12-1.module_el8+454+d 694 kB/s | 70 kB 00:00 2023-10-01T19:37:35.897 INFO:teuthology.orchestra.run.smithi057.stdout:(1/22): cockpit-podman-75-1.module_el8+664+4072 1.9 MB/s | 739 kB 00:00 2023-10-01T19:37:35.920 INFO:teuthology.orchestra.run.smithi087.stdout:(19/22): libslirp-4.4.0-1.module_el8+487+8e42a2 1.0 MB/s | 70 kB 00:00 2023-10-01T19:37:35.931 INFO:teuthology.orchestra.run.smithi057.stdout:(2/22): aardvark-dns-1.7.0-1.module_el8+487+8e4 2.4 MB/s | 1.0 MB 00:00 2023-10-01T19:37:35.964 INFO:teuthology.orchestra.run.smithi057.stdout:(3/22): criu-libs-3.18-4.module_el8+454+d7ef4b8 589 kB/s | 39 kB 00:00 2023-10-01T19:37:35.987 INFO:teuthology.orchestra.run.smithi087.stdout:(20/22): slirp4netns-1.2.1-1.module_el8+661+d1a 836 kB/s | 56 kB 00:00 2023-10-01T19:37:36.006 INFO:teuthology.orchestra.run.smithi057.stdout:(4/22): crun-1.8.7-1.module_el8+661+d1afb926.x8 3.1 MB/s | 239 kB 00:00 2023-10-01T19:37:36.073 INFO:teuthology.orchestra.run.smithi057.stdout:(5/22): python3-podman-4.6.0-1.module_el8+582+a 2.6 MB/s | 177 kB 00:00 2023-10-01T19:37:36.079 INFO:teuthology.orchestra.run.smithi087.stdout:(21/22): runc-1.1.9-1.module_el8+643+8db347f4.x 15 MB/s | 3.1 MB 00:00 2023-10-01T19:37:36.207 INFO:teuthology.orchestra.run.smithi057.stdout:(6/22): buildah-1.31.3-1.module_el8+664+4072b3a 13 MB/s | 8.8 MB 00:00 2023-10-01T19:37:36.232 INFO:teuthology.orchestra.run.smithi057.stdout:(7/22): python3-pytoml-0.1.14-5.git7dea353.el8. 156 kB/s | 25 kB 00:00 2023-10-01T19:37:36.291 INFO:teuthology.orchestra.run.smithi057.stdout:(8/22): netavark-1.7.0-1.module_el8+487+8e42a27 11 MB/s | 3.7 MB 00:00 2023-10-01T19:37:36.316 INFO:teuthology.orchestra.run.smithi057.stdout:(9/22): python3-pyxdg-0.25-16.el8.noarch.rpm 865 kB/s | 94 kB 00:00 2023-10-01T19:37:36.408 INFO:teuthology.orchestra.run.smithi057.stdout:(10/22): toolbox-0.0.99.4-5.module_el8+649+e2de 22 MB/s | 2.5 MB 00:00 2023-10-01T19:37:36.433 INFO:teuthology.orchestra.run.smithi057.stdout:(11/22): udica-0.2.6-20.module_el8+487+8e42a277 450 kB/s | 53 kB 00:00 2023-10-01T19:37:36.488 INFO:teuthology.orchestra.run.smithi087.stdout:(22/22): containernetworking-plugins-1.3.0-4.mo 25 MB/s | 21 MB 00:00 2023-10-01T19:37:36.489 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-10-01T19:37:36.489 INFO:teuthology.orchestra.run.smithi087.stdout:Total 26 MB/s | 51 MB 00:01 2023-10-01T19:37:36.492 INFO:teuthology.orchestra.run.smithi057.stdout:(12/22): yajl-2.1.0-12.el8.x86_64.rpm 489 kB/s | 41 kB 00:00 2023-10-01T19:37:36.517 INFO:teuthology.orchestra.run.smithi057.stdout:(13/22): conmon-2.1.8-1.module_el8+664+4072b3ae 687 kB/s | 57 kB 00:00 2023-10-01T19:37:36.559 INFO:teuthology.orchestra.run.smithi057.stdout:(14/22): container-selinux-2.221.0-1.module_el8 1.0 MB/s | 69 kB 00:00 2023-10-01T19:37:36.668 INFO:teuthology.orchestra.run.smithi057.stdout:(15/22): skopeo-1.13.3-1.module_el8+664+4072b3a 19 MB/s | 8.1 MB 00:00 2023-10-01T19:37:36.693 INFO:teuthology.orchestra.run.smithi057.stdout:(16/22): containers-common-1-54.module_el8+664+ 1.0 MB/s | 137 kB 00:00 2023-10-01T19:37:36.752 INFO:teuthology.orchestra.run.smithi057.stdout:(17/22): criu-3.18-4.module_el8+454+d7ef4b8d.x8 6.6 MB/s | 564 kB 00:00 2023-10-01T19:37:36.798 INFO:teuthology.orchestra.run.smithi057.stdout:(18/22): fuse-overlayfs-1.12-1.module_el8+454+d 668 kB/s | 70 kB 00:00 2023-10-01T19:37:36.836 INFO:teuthology.orchestra.run.smithi057.stdout:(19/22): libslirp-4.4.0-1.module_el8+487+8e42a2 844 kB/s | 70 kB 00:00 2023-10-01T19:37:36.903 INFO:teuthology.orchestra.run.smithi057.stdout:(20/22): slirp4netns-1.2.1-1.module_el8+661+d1a 837 kB/s | 56 kB 00:00 2023-10-01T19:37:36.978 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-10-01T19:37:37.003 INFO:teuthology.orchestra.run.smithi057.stdout:(21/22): runc-1.1.9-1.module_el8+643+8db347f4.x 15 MB/s | 3.1 MB 00:00 2023-10-01T19:37:37.025 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-10-01T19:37:37.025 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-10-01T19:37:37.361 INFO:teuthology.orchestra.run.smithi057.stdout:(22/22): containernetworking-plugins-1.3.0-4.mo 25 MB/s | 21 MB 00:00 2023-10-01T19:37:37.362 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2023-10-01T19:37:37.362 INFO:teuthology.orchestra.run.smithi057.stdout:Total 25 MB/s | 51 MB 00:02 2023-10-01T19:37:37.525 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-10-01T19:37:37.528 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-10-01T19:37:37.894 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2023-10-01T19:37:37.947 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2023-10-01T19:37:37.947 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2023-10-01T19:37:38.489 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2023-10-01T19:37:38.493 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2023-10-01T19:37:38.781 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-10-01T19:37:38.858 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: criu-3.18-4.module_el8+454+d7ef4b8d.x86_64 1/1 2023-10-01T19:37:39.078 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : criu-3.18-4.module_el8+454+d7ef4b8d.x86_64 1/31 2023-10-01T19:37:39.105 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: container-selinux-2:2.221.0-1.module_el8+653+feef7 2/31 2023-10-01T19:37:39.328 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : container-selinux-2:2.221.0-1.module_el8+653+feef7 2/31 2023-10-01T19:37:39.792 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2023-10-01T19:37:39.867 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: criu-3.18-4.module_el8+454+d7ef4b8d.x86_64 1/1 2023-10-01T19:37:40.054 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : criu-3.18-4.module_el8+454+d7ef4b8d.x86_64 1/31 2023-10-01T19:37:40.085 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: container-selinux-2:2.221.0-1.module_el8+653+feef7 2/31 2023-10-01T19:37:40.297 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : container-selinux-2:2.221.0-1.module_el8+653+feef7 2/31 2023-10-01T19:37:56.830 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: container-selinux-2:2.221.0-1.module_el8+653+feef7 2/31 2023-10-01T19:37:57.100 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : runc-1:1.1.9-1.module_el8+643+8db347f4.x86_64 3/31 2023-10-01T19:37:57.255 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : criu-libs-3.18-4.module_el8+454+d7ef4b8d.x86_64 4/31 2023-10-01T19:37:57.416 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : libslirp-4.4.0-1.module_el8+487+8e42a277.x86_64 5/31 2023-10-01T19:37:57.636 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : slirp4netns-1.2.1-1.module_el8+661+d1afb926.x86_64 6/31 2023-10-01T19:37:57.817 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : fuse-overlayfs-1.12-1.module_el8+454+d7ef4b8d.x86_ 7/31 2023-10-01T19:37:59.600 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: fuse-overlayfs-1.12-1.module_el8+454+d7ef4b8d.x86_ 7/31 2023-10-01T19:37:59.816 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : containernetworking-plugins-1:1.3.0-4.module_el8+6 8/31 2023-10-01T19:38:00.053 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : yajl-2.1.0-12.el8.x86_64 9/31 2023-10-01T19:38:00.240 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : crun-1.8.7-1.module_el8+661+d1afb926.x86_64 10/31 2023-10-01T19:38:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : containers-common-2:1-54.module_el8+664+4072b3ae.x 11/31 2023-10-01T19:38:00.700 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 12/31 2023-10-01T19:38:00.909 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: container-selinux-2:2.221.0-1.module_el8+653+feef7 2/31 2023-10-01T19:38:01.003 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 13/31 2023-10-01T19:38:01.141 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : runc-1:1.1.9-1.module_el8+643+8db347f4.x86_64 3/31 2023-10-01T19:38:01.279 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : criu-libs-3.18-4.module_el8+454+d7ef4b8d.x86_64 4/31 2023-10-01T19:38:01.494 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libslirp-4.4.0-1.module_el8+487+8e42a277.x86_64 5/31 2023-10-01T19:38:01.605 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : aardvark-dns-2:1.7.0-1.module_el8+487+8e42a277.x86 14/31 2023-10-01T19:38:01.706 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : slirp4netns-1.2.1-1.module_el8+661+d1afb926.x86_64 6/31 2023-10-01T19:38:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : fuse-overlayfs-1.12-1.module_el8+454+d7ef4b8d.x86_ 7/31 2023-10-01T19:38:02.491 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : netavark-2:1.7.0-1.module_el8+487+8e42a277.x86_64 15/31 2023-10-01T19:38:02.773 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : buildah-1:1.31.3-1.module_el8+664+4072b3ae.x86_64 16/31 2023-10-01T19:38:03.717 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-podman-4.6.0-1.module_el8+582+a2558ae2.noa 17/31 2023-10-01T19:38:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: fuse-overlayfs-1.12-1.module_el8+454+d7ef4b8d.x86_ 7/31 2023-10-01T19:38:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : containernetworking-plugins-1:1.3.0-4.module_el8+6 8/31 2023-10-01T19:38:04.132 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : skopeo-2:1.13.3-1.module_el8+664+4072b3ae.x86_64 18/31 2023-10-01T19:38:04.349 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : yajl-2.1.0-12.el8.x86_64 9/31 2023-10-01T19:38:04.353 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : toolbox-0.0.99.4-5.module_el8+649+e2deac1d.x86_64 19/31 2023-10-01T19:38:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : udica-0.2.6-20.module_el8+487+8e42a277.noarch 20/31 2023-10-01T19:38:04.568 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : crun-1.8.7-1.module_el8+661+d1afb926.x86_64 10/31 2023-10-01T19:38:04.807 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : conmon-3:2.1.8-1.module_el8+664+4072b3ae.x86_64 21/31 2023-10-01T19:38:04.825 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : containers-common-2:1-54.module_el8+664+4072b3ae.x 11/31 2023-10-01T19:38:04.976 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cockpit-podman-75-1.module_el8+664+4072b3ae.noarch 22/31 2023-10-01T19:38:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 12/31 2023-10-01T19:38:05.149 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : containers-common-2:1-49.module_el8.8.0+1254+78119 23/31 2023-10-01T19:38:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 13/31 2023-10-01T19:38:05.309 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : slirp4netns-1.2.0-2.module_el8.7.0+1216+b022c01d.x 24/31 2023-10-01T19:38:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : runc-1:1.1.4-1.module_el8.7.0+1216+b022c01d.x86_64 25/31 2023-10-01T19:38:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : container-selinux-2:2.195.1-1.module_el8.8.0+1254+ 26/31 2023-10-01T19:38:05.676 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: container-selinux-2:2.195.1-1.module_el8.8.0+1254+ 26/31 2023-10-01T19:38:05.880 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : aardvark-dns-2:1.7.0-1.module_el8+487+8e42a277.x86 14/31 2023-10-01T19:38:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : criu-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 27/31 2023-10-01T19:38:06.059 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : libslirp-4.4.0-1.module_el8.7.0+1216+b022c01d.x86_ 28/31 2023-10-01T19:38:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : fuse-overlayfs-1.10-1.module_el8.8.0+1254+78119b6e 29/31 2023-10-01T19:38:06.517 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : containernetworking-plugins-1:1.1.1-3.module_el8.7 30/31 2023-10-01T19:38:06.716 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : conmon-3:2.1.5-1.module_el8.8.0+1254+78119b6e.x86_ 31/31 2023-10-01T19:38:06.851 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : netavark-2:1.7.0-1.module_el8+487+8e42a277.x86_64 15/31 2023-10-01T19:38:07.134 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : buildah-1:1.31.3-1.module_el8+664+4072b3ae.x86_64 16/31 2023-10-01T19:38:07.169 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: container-selinux-2:2.221.0-1.module_el8+653+feef7 31/31 2023-10-01T19:38:07.615 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: conmon-3:2.1.5-1.module_el8.8.0+1254+78119b6e.x86_ 31/31 2023-10-01T19:38:07.615 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : aardvark-dns-2:1.7.0-1.module_el8+487+8e42a277.x86 1/31 2023-10-01T19:38:07.615 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : buildah-1:1.31.3-1.module_el8+664+4072b3ae.x86_64 2/31 2023-10-01T19:38:07.615 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cockpit-podman-75-1.module_el8+664+4072b3ae.noarch 3/31 2023-10-01T19:38:07.615 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : criu-libs-3.18-4.module_el8+454+d7ef4b8d.x86_64 4/31 2023-10-01T19:38:07.615 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : crun-1.8.7-1.module_el8+661+d1afb926.x86_64 5/31 2023-10-01T19:38:07.615 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : netavark-2:1.7.0-1.module_el8+487+8e42a277.x86_64 6/31 2023-10-01T19:38:07.615 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-podman-4.6.0-1.module_el8+582+a2558ae2.noa 7/31 2023-10-01T19:38:07.615 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/31 2023-10-01T19:38:07.616 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/31 2023-10-01T19:38:07.616 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : skopeo-2:1.13.3-1.module_el8+664+4072b3ae.x86_64 10/31 2023-10-01T19:38:07.616 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : toolbox-0.0.99.4-5.module_el8+649+e2deac1d.x86_64 11/31 2023-10-01T19:38:07.616 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : udica-0.2.6-20.module_el8+487+8e42a277.noarch 12/31 2023-10-01T19:38:07.616 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/31 2023-10-01T19:38:07.616 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : conmon-3:2.1.8-1.module_el8+664+4072b3ae.x86_64 14/31 2023-10-01T19:38:07.616 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : conmon-3:2.1.5-1.module_el8.8.0+1254+78119b6e.x86_ 15/31 2023-10-01T19:38:07.616 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : container-selinux-2:2.221.0-1.module_el8+653+feef7 16/31 2023-10-01T19:38:07.616 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : container-selinux-2:2.195.1-1.module_el8.8.0+1254+ 17/31 2023-10-01T19:38:07.616 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : containernetworking-plugins-1:1.3.0-4.module_el8+6 18/31 2023-10-01T19:38:07.616 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : containernetworking-plugins-1:1.1.1-3.module_el8.7 19/31 2023-10-01T19:38:07.617 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : containers-common-2:1-54.module_el8+664+4072b3ae.x 20/31 2023-10-01T19:38:07.617 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : containers-common-2:1-49.module_el8.8.0+1254+78119 21/31 2023-10-01T19:38:07.617 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : criu-3.18-4.module_el8+454+d7ef4b8d.x86_64 22/31 2023-10-01T19:38:07.617 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : criu-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 23/31 2023-10-01T19:38:07.617 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : fuse-overlayfs-1.12-1.module_el8+454+d7ef4b8d.x86_ 24/31 2023-10-01T19:38:07.617 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : fuse-overlayfs-1.10-1.module_el8.8.0+1254+78119b6e 25/31 2023-10-01T19:38:07.617 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libslirp-4.4.0-1.module_el8+487+8e42a277.x86_64 26/31 2023-10-01T19:38:07.617 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libslirp-4.4.0-1.module_el8.7.0+1216+b022c01d.x86_ 27/31 2023-10-01T19:38:07.617 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : runc-1:1.1.9-1.module_el8+643+8db347f4.x86_64 28/31 2023-10-01T19:38:07.617 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : runc-1:1.1.4-1.module_el8.7.0+1216+b022c01d.x86_64 29/31 2023-10-01T19:38:07.617 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : slirp4netns-1.2.1-1.module_el8+661+d1afb926.x86_64 30/31 2023-10-01T19:38:08.162 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-podman-4.6.0-1.module_el8+582+a2558ae2.noa 17/31 2023-10-01T19:38:08.534 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : slirp4netns-1.2.0-2.module_el8.7.0+1216+b022c01d.x 31/31 2023-10-01T19:38:08.534 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:38:08.534 INFO:teuthology.orchestra.run.smithi087.stdout:Upgraded: 2023-10-01T19:38:08.534 INFO:teuthology.orchestra.run.smithi087.stdout: conmon-3:2.1.8-1.module_el8+664+4072b3ae.x86_64 2023-10-01T19:38:08.534 INFO:teuthology.orchestra.run.smithi087.stdout: container-selinux-2:2.221.0-1.module_el8+653+feef7bfe.noarch 2023-10-01T19:38:08.534 INFO:teuthology.orchestra.run.smithi087.stdout: containernetworking-plugins-1:1.3.0-4.module_el8+644+cb8a8b65.x86_64 2023-10-01T19:38:08.535 INFO:teuthology.orchestra.run.smithi087.stdout: containers-common-2:1-54.module_el8+664+4072b3ae.x86_64 2023-10-01T19:38:08.535 INFO:teuthology.orchestra.run.smithi087.stdout: criu-3.18-4.module_el8+454+d7ef4b8d.x86_64 2023-10-01T19:38:08.535 INFO:teuthology.orchestra.run.smithi087.stdout: fuse-overlayfs-1.12-1.module_el8+454+d7ef4b8d.x86_64 2023-10-01T19:38:08.535 INFO:teuthology.orchestra.run.smithi087.stdout: libslirp-4.4.0-1.module_el8+487+8e42a277.x86_64 2023-10-01T19:38:08.535 INFO:teuthology.orchestra.run.smithi087.stdout: runc-1:1.1.9-1.module_el8+643+8db347f4.x86_64 2023-10-01T19:38:08.535 INFO:teuthology.orchestra.run.smithi087.stdout: slirp4netns-1.2.1-1.module_el8+661+d1afb926.x86_64 2023-10-01T19:38:08.535 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-10-01T19:38:08.535 INFO:teuthology.orchestra.run.smithi087.stdout: aardvark-dns-2:1.7.0-1.module_el8+487+8e42a277.x86_64 2023-10-01T19:38:08.535 INFO:teuthology.orchestra.run.smithi087.stdout: buildah-1:1.31.3-1.module_el8+664+4072b3ae.x86_64 2023-10-01T19:38:08.535 INFO:teuthology.orchestra.run.smithi087.stdout: cockpit-podman-75-1.module_el8+664+4072b3ae.noarch 2023-10-01T19:38:08.535 INFO:teuthology.orchestra.run.smithi087.stdout: criu-libs-3.18-4.module_el8+454+d7ef4b8d.x86_64 2023-10-01T19:38:08.535 INFO:teuthology.orchestra.run.smithi087.stdout: crun-1.8.7-1.module_el8+661+d1afb926.x86_64 2023-10-01T19:38:08.536 INFO:teuthology.orchestra.run.smithi087.stdout: netavark-2:1.7.0-1.module_el8+487+8e42a277.x86_64 2023-10-01T19:38:08.536 INFO:teuthology.orchestra.run.smithi087.stdout: python3-podman-4.6.0-1.module_el8+582+a2558ae2.noarch 2023-10-01T19:38:08.536 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-01T19:38:08.536 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-01T19:38:08.536 INFO:teuthology.orchestra.run.smithi087.stdout: skopeo-2:1.13.3-1.module_el8+664+4072b3ae.x86_64 2023-10-01T19:38:08.536 INFO:teuthology.orchestra.run.smithi087.stdout: toolbox-0.0.99.4-5.module_el8+649+e2deac1d.x86_64 2023-10-01T19:38:08.536 INFO:teuthology.orchestra.run.smithi087.stdout: udica-0.2.6-20.module_el8+487+8e42a277.noarch 2023-10-01T19:38:08.536 INFO:teuthology.orchestra.run.smithi087.stdout: yajl-2.1.0-12.el8.x86_64 2023-10-01T19:38:08.536 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:38:08.536 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-10-01T19:38:08.585 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : skopeo-2:1.13.3-1.module_el8+664+4072b3ae.x86_64 18/31 2023-10-01T19:38:08.775 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : toolbox-0.0.99.4-5.module_el8+649+e2deac1d.x86_64 19/31 2023-10-01T19:38:08.843 DEBUG:teuthology.parallel:result is None 2023-10-01T19:38:08.976 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : udica-0.2.6-20.module_el8+487+8e42a277.noarch 20/31 2023-10-01T19:38:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : conmon-3:2.1.8-1.module_el8+664+4072b3ae.x86_64 21/31 2023-10-01T19:38:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cockpit-podman-75-1.module_el8+664+4072b3ae.noarch 22/31 2023-10-01T19:38:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : containers-common-2:1-49.module_el8.8.0+1254+78119 23/31 2023-10-01T19:38:09.711 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : slirp4netns-1.2.0-2.module_el8.7.0+1216+b022c01d.x 24/31 2023-10-01T19:38:09.861 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : runc-1:1.1.4-1.module_el8.7.0+1216+b022c01d.x86_64 25/31 2023-10-01T19:38:09.861 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : container-selinux-2:2.195.1-1.module_el8.8.0+1254+ 26/31 2023-10-01T19:38:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: container-selinux-2:2.195.1-1.module_el8.8.0+1254+ 26/31 2023-10-01T19:38:10.275 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : criu-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 27/31 2023-10-01T19:38:10.427 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libslirp-4.4.0-1.module_el8.7.0+1216+b022c01d.x86_ 28/31 2023-10-01T19:38:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : fuse-overlayfs-1.10-1.module_el8.8.0+1254+78119b6e 29/31 2023-10-01T19:38:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : containernetworking-plugins-1:1.1.1-3.module_el8.7 30/31 2023-10-01T19:38:11.026 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : conmon-3:2.1.5-1.module_el8.8.0+1254+78119b6e.x86_ 31/31 2023-10-01T19:38:11.547 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: container-selinux-2:2.221.0-1.module_el8+653+feef7 31/31 2023-10-01T19:38:12.059 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: conmon-3:2.1.5-1.module_el8.8.0+1254+78119b6e.x86_ 31/31 2023-10-01T19:38:12.059 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : aardvark-dns-2:1.7.0-1.module_el8+487+8e42a277.x86 1/31 2023-10-01T19:38:12.059 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : buildah-1:1.31.3-1.module_el8+664+4072b3ae.x86_64 2/31 2023-10-01T19:38:12.059 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cockpit-podman-75-1.module_el8+664+4072b3ae.noarch 3/31 2023-10-01T19:38:12.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : criu-libs-3.18-4.module_el8+454+d7ef4b8d.x86_64 4/31 2023-10-01T19:38:12.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : crun-1.8.7-1.module_el8+661+d1afb926.x86_64 5/31 2023-10-01T19:38:12.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : netavark-2:1.7.0-1.module_el8+487+8e42a277.x86_64 6/31 2023-10-01T19:38:12.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-podman-4.6.0-1.module_el8+582+a2558ae2.noa 7/31 2023-10-01T19:38:12.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/31 2023-10-01T19:38:12.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/31 2023-10-01T19:38:12.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : skopeo-2:1.13.3-1.module_el8+664+4072b3ae.x86_64 10/31 2023-10-01T19:38:12.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : toolbox-0.0.99.4-5.module_el8+649+e2deac1d.x86_64 11/31 2023-10-01T19:38:12.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : udica-0.2.6-20.module_el8+487+8e42a277.noarch 12/31 2023-10-01T19:38:12.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/31 2023-10-01T19:38:12.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : conmon-3:2.1.8-1.module_el8+664+4072b3ae.x86_64 14/31 2023-10-01T19:38:12.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : conmon-3:2.1.5-1.module_el8.8.0+1254+78119b6e.x86_ 15/31 2023-10-01T19:38:12.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : container-selinux-2:2.221.0-1.module_el8+653+feef7 16/31 2023-10-01T19:38:12.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : container-selinux-2:2.195.1-1.module_el8.8.0+1254+ 17/31 2023-10-01T19:38:12.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : containernetworking-plugins-1:1.3.0-4.module_el8+6 18/31 2023-10-01T19:38:12.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : containernetworking-plugins-1:1.1.1-3.module_el8.7 19/31 2023-10-01T19:38:12.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : containers-common-2:1-54.module_el8+664+4072b3ae.x 20/31 2023-10-01T19:38:12.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : containers-common-2:1-49.module_el8.8.0+1254+78119 21/31 2023-10-01T19:38:12.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : criu-3.18-4.module_el8+454+d7ef4b8d.x86_64 22/31 2023-10-01T19:38:12.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : criu-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 23/31 2023-10-01T19:38:12.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : fuse-overlayfs-1.12-1.module_el8+454+d7ef4b8d.x86_ 24/31 2023-10-01T19:38:12.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : fuse-overlayfs-1.10-1.module_el8.8.0+1254+78119b6e 25/31 2023-10-01T19:38:12.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libslirp-4.4.0-1.module_el8+487+8e42a277.x86_64 26/31 2023-10-01T19:38:12.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libslirp-4.4.0-1.module_el8.7.0+1216+b022c01d.x86_ 27/31 2023-10-01T19:38:12.062 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : runc-1:1.1.9-1.module_el8+643+8db347f4.x86_64 28/31 2023-10-01T19:38:12.062 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : runc-1:1.1.4-1.module_el8.7.0+1216+b022c01d.x86_64 29/31 2023-10-01T19:38:12.062 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : slirp4netns-1.2.1-1.module_el8+661+d1afb926.x86_64 30/31 2023-10-01T19:38:13.078 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : slirp4netns-1.2.0-2.module_el8.7.0+1216+b022c01d.x 31/31 2023-10-01T19:38:13.078 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:38:13.079 INFO:teuthology.orchestra.run.smithi057.stdout:Upgraded: 2023-10-01T19:38:13.079 INFO:teuthology.orchestra.run.smithi057.stdout: conmon-3:2.1.8-1.module_el8+664+4072b3ae.x86_64 2023-10-01T19:38:13.079 INFO:teuthology.orchestra.run.smithi057.stdout: container-selinux-2:2.221.0-1.module_el8+653+feef7bfe.noarch 2023-10-01T19:38:13.079 INFO:teuthology.orchestra.run.smithi057.stdout: containernetworking-plugins-1:1.3.0-4.module_el8+644+cb8a8b65.x86_64 2023-10-01T19:38:13.079 INFO:teuthology.orchestra.run.smithi057.stdout: containers-common-2:1-54.module_el8+664+4072b3ae.x86_64 2023-10-01T19:38:13.079 INFO:teuthology.orchestra.run.smithi057.stdout: criu-3.18-4.module_el8+454+d7ef4b8d.x86_64 2023-10-01T19:38:13.079 INFO:teuthology.orchestra.run.smithi057.stdout: fuse-overlayfs-1.12-1.module_el8+454+d7ef4b8d.x86_64 2023-10-01T19:38:13.079 INFO:teuthology.orchestra.run.smithi057.stdout: libslirp-4.4.0-1.module_el8+487+8e42a277.x86_64 2023-10-01T19:38:13.079 INFO:teuthology.orchestra.run.smithi057.stdout: runc-1:1.1.9-1.module_el8+643+8db347f4.x86_64 2023-10-01T19:38:13.079 INFO:teuthology.orchestra.run.smithi057.stdout: slirp4netns-1.2.1-1.module_el8+661+d1afb926.x86_64 2023-10-01T19:38:13.079 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2023-10-01T19:38:13.080 INFO:teuthology.orchestra.run.smithi057.stdout: aardvark-dns-2:1.7.0-1.module_el8+487+8e42a277.x86_64 2023-10-01T19:38:13.080 INFO:teuthology.orchestra.run.smithi057.stdout: buildah-1:1.31.3-1.module_el8+664+4072b3ae.x86_64 2023-10-01T19:38:13.080 INFO:teuthology.orchestra.run.smithi057.stdout: cockpit-podman-75-1.module_el8+664+4072b3ae.noarch 2023-10-01T19:38:13.080 INFO:teuthology.orchestra.run.smithi057.stdout: criu-libs-3.18-4.module_el8+454+d7ef4b8d.x86_64 2023-10-01T19:38:13.080 INFO:teuthology.orchestra.run.smithi057.stdout: crun-1.8.7-1.module_el8+661+d1afb926.x86_64 2023-10-01T19:38:13.080 INFO:teuthology.orchestra.run.smithi057.stdout: netavark-2:1.7.0-1.module_el8+487+8e42a277.x86_64 2023-10-01T19:38:13.080 INFO:teuthology.orchestra.run.smithi057.stdout: python3-podman-4.6.0-1.module_el8+582+a2558ae2.noarch 2023-10-01T19:38:13.080 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-01T19:38:13.080 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-01T19:38:13.080 INFO:teuthology.orchestra.run.smithi057.stdout: skopeo-2:1.13.3-1.module_el8+664+4072b3ae.x86_64 2023-10-01T19:38:13.080 INFO:teuthology.orchestra.run.smithi057.stdout: toolbox-0.0.99.4-5.module_el8+649+e2deac1d.x86_64 2023-10-01T19:38:13.080 INFO:teuthology.orchestra.run.smithi057.stdout: udica-0.2.6-20.module_el8+487+8e42a277.noarch 2023-10-01T19:38:13.081 INFO:teuthology.orchestra.run.smithi057.stdout: yajl-2.1.0-12.el8.x86_64 2023-10-01T19:38:13.081 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:38:13.081 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-10-01T19:38:13.466 DEBUG:teuthology.parallel:result is None 2023-10-01T19:38:13.467 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-01T19:38:13.537 INFO:tasks.cephadm:Config: {'cephadm_branch': 'v17.2.0', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'image': 'quay.io/ceph/ceph:v17.2.0', 'conf': {'global': {'mon election default strategy': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': True}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '37d71fd39f09a164b3518d8aef9e20133197e5f9'} 2023-10-01T19:38:13.537 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v17.2.0 2023-10-01T19:38:13.538 INFO:tasks.cephadm:Cluster fsid is 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:38:13.538 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-01T19:38:13.538 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.57', 'mon.c': '[v2:172.21.15.57:3301,v1:172.21.15.57:6790]', 'mon.b': '172.21.15.87'} 2023-10-01T19:38:13.538 INFO:tasks.cephadm:First mon is mon.a on smithi057 2023-10-01T19:38:13.538 INFO:tasks.cephadm:First mgr is y 2023-10-01T19:38:13.538 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-01T19:38:13.539 DEBUG:teuthology.orchestra.run.smithi057:> sudo hostname $(hostname -s) 2023-10-01T19:38:13.573 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname $(hostname -s) 2023-10-01T19:38:13.607 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref v17.2.0)... 2023-10-01T19:38:13.607 DEBUG:teuthology.orchestra.run.smithi057:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v17.2.0/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-01T19:38:13.858 INFO:teuthology.orchestra.run.smithi057.stdout:-rw-rw-r--. 1 ubuntu ubuntu 320521 Oct 1 19:38 /home/ubuntu/cephtest/cephadm 2023-10-01T19:38:13.858 DEBUG:teuthology.orchestra.run.smithi087:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v17.2.0/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-01T19:38:14.127 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-rw-r--. 1 ubuntu ubuntu 320521 Oct 1 19:38 /home/ubuntu/cephtest/cephadm 2023-10-01T19:38:14.127 DEBUG:teuthology.orchestra.run.smithi057:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-01T19:38:14.152 DEBUG:teuthology.orchestra.run.smithi087:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-01T19:38:14.184 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v17.2.0 on all hosts... 2023-10-01T19:38:14.184 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 pull 2023-10-01T19:38:14.194 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 pull 2023-10-01T19:38:15.942 INFO:teuthology.orchestra.run.smithi087.stderr:Pulling container image quay.io/ceph/ceph:v17.2.0... 2023-10-01T19:38:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:Pulling container image quay.io/ceph/ceph:v17.2.0... 2023-10-01T19:38:32.444 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2023-10-01T19:38:32.444 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)", 2023-10-01T19:38:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: "image_id": "e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9", 2023-10-01T19:38:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: "repo_digests": [ 2023-10-01T19:38:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: "quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a", 2023-10-01T19:38:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: "quay.io/ceph/ceph@sha256:cb4d698cb769b6aba05bf6ef04f41a7fe694160140347576e13bd9348514b667" 2023-10-01T19:38:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2023-10-01T19:38:32.445 INFO:teuthology.orchestra.run.smithi087.stdout:} 2023-10-01T19:38:33.949 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T19:38:33.949 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)", 2023-10-01T19:38:33.949 INFO:teuthology.orchestra.run.smithi057.stdout: "image_id": "e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9", 2023-10-01T19:38:33.949 INFO:teuthology.orchestra.run.smithi057.stdout: "repo_digests": [ 2023-10-01T19:38:33.949 INFO:teuthology.orchestra.run.smithi057.stdout: "quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a", 2023-10-01T19:38:33.949 INFO:teuthology.orchestra.run.smithi057.stdout: "quay.io/ceph/ceph@sha256:cb4d698cb769b6aba05bf6ef04f41a7fe694160140347576e13bd9348514b667" 2023-10-01T19:38:33.949 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-01T19:38:33.949 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T19:38:33.974 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /etc/ceph 2023-10-01T19:38:34.012 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph 2023-10-01T19:38:34.050 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 777 /etc/ceph 2023-10-01T19:38:34.089 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 777 /etc/ceph 2023-10-01T19:38:34.121 INFO:tasks.cephadm:Writing seed config... 2023-10-01T19:38:34.122 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2023-10-01T19:38:34.122 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-01T19:38:34.123 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-01T19:38:34.123 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = True 2023-10-01T19:38:34.123 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-01T19:38:34.123 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-01T19:38:34.123 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-01T19:38:34.123 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-01T19:38:34.123 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-01T19:38:34.124 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-01T19:38:34.124 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-01T19:38:34.153 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = True [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2023-10-01T19:38:34.153 DEBUG:teuthology.orchestra.run.smithi057:mon.a> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.a.service 2023-10-01T19:38:34.196 DEBUG:teuthology.orchestra.run.smithi057:mgr.y> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.y.service 2023-10-01T19:38:34.232 INFO:journalctl@ceph.mon.a.smithi057.stdout:-- Logs begin at Sun 2023-10-01 19:27:22 UTC. -- 2023-10-01T19:38:34.233 INFO:tasks.cephadm:Bootstrapping... 2023-10-01T19:38:34.233 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 -v bootstrap --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.57 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-01T19:38:34.266 INFO:journalctl@ceph.mgr.y.smithi057.stdout:-- Logs begin at Sun 2023-10-01 19:27:22 UTC. -- 2023-10-01T19:38:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:-------------------------------------------------------------------------------- 2023-10-01T19:38:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:cephadm ['--image', 'quay.io/ceph/ceph:v17.2.0', '-v', 'bootstrap', '--fsid', '0f6d5dc0-6092-11ee-8db5-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.57', '--skip-admin-label'] 2023-10-01T19:38:34.484 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: 4.6.1 2023-10-01T19:38:34.497 INFO:teuthology.orchestra.run.smithi057.stderr:Verifying podman|docker is present... 2023-10-01T19:38:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: 4.6.1 2023-10-01T19:38:34.532 INFO:teuthology.orchestra.run.smithi057.stderr:Verifying lvm2 is present... 2023-10-01T19:38:34.532 INFO:teuthology.orchestra.run.smithi057.stderr:Verifying time synchronization is in place... 2023-10-01T19:38:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2023-10-01T19:38:34.546 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: inactive 2023-10-01T19:38:34.553 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: enabled 2023-10-01T19:38:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: active 2023-10-01T19:38:34.560 INFO:teuthology.orchestra.run.smithi057.stderr:Unit chronyd.service is enabled and running 2023-10-01T19:38:34.560 INFO:teuthology.orchestra.run.smithi057.stderr:Repeating the final host check... 2023-10-01T19:38:34.582 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: 4.6.1 2023-10-01T19:38:34.591 INFO:teuthology.orchestra.run.smithi057.stderr:podman (/bin/podman) version 4.6.1 is present 2023-10-01T19:38:34.591 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl is present 2023-10-01T19:38:34.591 INFO:teuthology.orchestra.run.smithi057.stderr:lvcreate is present 2023-10-01T19:38:34.598 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2023-10-01T19:38:34.604 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: inactive 2023-10-01T19:38:34.611 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: enabled 2023-10-01T19:38:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: active 2023-10-01T19:38:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:Unit chronyd.service is enabled and running 2023-10-01T19:38:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:Host looks OK 2023-10-01T19:38:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:Cluster fsid: 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:38:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:Acquiring lock 140655305261184 on /run/cephadm/0f6d5dc0-6092-11ee-8db5-212e2dc638e7.lock 2023-10-01T19:38:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:Lock 140655305261184 acquired on /run/cephadm/0f6d5dc0-6092-11ee-8db5-212e2dc638e7.lock 2023-10-01T19:38:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:Verifying IP 172.21.15.57 port 3300 ... 2023-10-01T19:38:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:Verifying IP 172.21.15.57 port 6789 ... 2023-10-01T19:38:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:Base mon IP is 172.21.15.57, final addrv is [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2023-10-01T19:38:34.622 INFO:teuthology.orchestra.run.smithi057.stderr:/sbin/ip: default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.57 metric 100 2023-10-01T19:38:34.622 INFO:teuthology.orchestra.run.smithi057.stderr:/sbin/ip: 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.57 metric 100 2023-10-01T19:38:34.625 INFO:teuthology.orchestra.run.smithi057.stderr:/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2023-10-01T19:38:34.625 INFO:teuthology.orchestra.run.smithi057.stderr:/sbin/ip: fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-10-01T19:38:34.626 INFO:teuthology.orchestra.run.smithi057.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1689sec hoplimit 64 pref medium 2023-10-01T19:38:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-01T19:38:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:/sbin/ip: inet6 ::1/128 scope host 2023-10-01T19:38:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2023-10-01T19:38:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:/sbin/ip: 4: ens1f0: mtu 1500 state UP qlen 1000 2023-10-01T19:38:34.629 INFO:teuthology.orchestra.run.smithi057.stderr:/sbin/ip: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-10-01T19:38:34.629 INFO:teuthology.orchestra.run.smithi057.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2023-10-01T19:38:34.629 INFO:teuthology.orchestra.run.smithi057.stderr:Mon IP `172.21.15.57` is in CIDR network `172.21.0.0/20` 2023-10-01T19:38:34.630 INFO:teuthology.orchestra.run.smithi057.stderr:- internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-01T19:38:34.630 INFO:teuthology.orchestra.run.smithi057.stderr:Pulling container image quay.io/ceph/ceph:v17.2.0... 2023-10-01T19:38:34.744 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: Trying to pull quay.io/ceph/ceph:v17.2.0... 2023-10-01T19:38:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: Getting image source signatures 2023-10-01T19:38:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: Copying blob sha256:a70843738bb77e1ab9c1f85969ebdfa55f178e746be081d1cb4f94011f69eb7c 2023-10-01T19:38:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: Copying blob sha256:33ca8fff7868c4dc0c11e09bca97c720eb9cfbab7221216754367dd8de70388a 2023-10-01T19:38:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: Copying blob sha256:c32ab78b488d0c72f64eded765c0cf6b5bf2c75dab66cb62a9d367fa6ec42513 2023-10-01T19:38:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: Copying blob sha256:89b4a75bc2d8500f15463747507c9623df43886c134463e7f0527e70900e7a7b 2023-10-01T19:38:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: Copying blob sha256:599d07cb321ff0a3c82224e1138fc685793fa69b93ed5780415751a5f7e4b8c2 2023-10-01T19:38:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: Copying config sha256:e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 2023-10-01T19:38:35.249 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: Writing manifest to image destination 2023-10-01T19:38:35.412 INFO:teuthology.orchestra.run.smithi057.stderr:/bin/podman: e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 2023-10-01T19:38:35.854 INFO:teuthology.orchestra.run.smithi057.stderr:ceph: ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable) 2023-10-01T19:38:36.212 INFO:teuthology.orchestra.run.smithi057.stderr:Ceph version: ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable) 2023-10-01T19:38:36.213 INFO:teuthology.orchestra.run.smithi057.stderr:Extracting ceph user uid/gid from container image... 2023-10-01T19:38:36.512 INFO:teuthology.orchestra.run.smithi057.stderr:stat: 167 167 2023-10-01T19:38:36.787 INFO:teuthology.orchestra.run.smithi057.stderr:Creating initial keys... 2023-10-01T19:38:37.102 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph-authtool: AQC9yhllxo7lBBAAupkGmE5v+5/6zdFm3EhTSw== 2023-10-01T19:38:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph-authtool: AQC9yhllsdudKBAAdnJaVXj00wso4ZLC6K5Gyw== 2023-10-01T19:38:38.318 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph-authtool: AQC+yhllzM1NERAAn33GO0C5Lfq+hpl+hhRYIA== 2023-10-01T19:38:38.632 INFO:teuthology.orchestra.run.smithi057.stderr:Creating initial monmap... 2023-10-01T19:38:38.944 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-01T19:38:38.945 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/monmaptool: setting min_mon_release = octopus 2023-10-01T19:38:38.945 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:38:38.945 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-01T19:38:39.282 INFO:teuthology.orchestra.run.smithi057.stderr:monmaptool for a [v2:172.21.15.57:3300,v1:172.21.15.57:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-01T19:38:39.282 INFO:teuthology.orchestra.run.smithi057.stderr:setting min_mon_release = octopus 2023-10-01T19:38:39.282 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/monmaptool: set fsid to 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:38:39.282 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-01T19:38:39.283 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-01T19:38:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:Creating mon... 2023-10-01T19:38:39.991 INFO:teuthology.orchestra.run.smithi057.stderr:create mon.a on 2023-10-01T19:38:40.118 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-01T19:38:40.222 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7.target. 2023-10-01T19:38:40.222 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7.target. 2023-10-01T19:38:40.455 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: Failed to reset failed state of unit ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.a.service: Unit ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.a.service not loaded. 2023-10-01T19:38:40.462 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: Created symlink /etc/systemd/system/ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7.target.wants/ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@.service. 2023-10-01T19:38:40.980 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: disabled 2023-10-01T19:38:40.989 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: inactive 2023-10-01T19:38:40.990 INFO:teuthology.orchestra.run.smithi057.stderr:firewalld.service is not enabled 2023-10-01T19:38:40.990 INFO:teuthology.orchestra.run.smithi057.stderr:Not possible to enable service . firewalld.service is not available 2023-10-01T19:38:40.990 INFO:teuthology.orchestra.run.smithi057.stderr:Waiting for mon to start... 2023-10-01T19:38:40.991 INFO:teuthology.orchestra.run.smithi057.stderr:Waiting for mon... 2023-10-01T19:38:41.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:40 smithi057 ceph-mon[107832]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-01T19:38:41.530 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: cluster: 2023-10-01T19:38:41.530 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: id: 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:38:41.530 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: health: HEALTH_OK 2023-10-01T19:38:41.530 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:41.530 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: services: 2023-10-01T19:38:41.530 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: mon: 1 daemons, quorum a (age 0.560643s) 2023-10-01T19:38:41.530 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: mgr: no daemons active 2023-10-01T19:38:41.530 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2023-10-01T19:38:41.530 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:41.530 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: data: 2023-10-01T19:38:41.531 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2023-10-01T19:38:41.531 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2023-10-01T19:38:41.531 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2023-10-01T19:38:41.531 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: pgs: 2023-10-01T19:38:41.531 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:41.901 INFO:teuthology.orchestra.run.smithi057.stderr:mon is available 2023-10-01T19:38:41.901 INFO:teuthology.orchestra.run.smithi057.stderr:Assimilating anything we can from ceph.conf... 2023-10-01T19:38:42.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:41 smithi057 ceph-mon[107832]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-01T19:38:42.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:41 smithi057 ceph-mon[107832]: monmap e1: 1 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0]} 2023-10-01T19:38:42.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:41 smithi057 ceph-mon[107832]: fsmap 2023-10-01T19:38:42.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:41 smithi057 ceph-mon[107832]: osdmap e1: 0 total, 0 up, 0 in 2023-10-01T19:38:42.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:41 smithi057 ceph-mon[107832]: mgrmap e1: no daemons active 2023-10-01T19:38:42.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:41 smithi057 ceph-mon[107832]: from='client.? 172.21.15.57:0/2570215816' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T19:38:42.427 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:42.427 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: [global] 2023-10-01T19:38:42.427 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: fsid = 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:38:42.427 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2023-10-01T19:38:42.427 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2023-10-01T19:38:42.427 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2023-10-01T19:38:42.428 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2023-10-01T19:38:42.428 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2023-10-01T19:38:42.428 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:42.428 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: [mgr] 2023-10-01T19:38:42.428 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: mgr/cephadm/use_agent = True 2023-10-01T19:38:42.428 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2023-10-01T19:38:42.428 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:42.428 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: [osd] 2023-10-01T19:38:42.428 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2023-10-01T19:38:42.428 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2023-10-01T19:38:42.771 INFO:teuthology.orchestra.run.smithi057.stderr:Generating new minimal ceph.conf... 2023-10-01T19:38:43.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:42 smithi057 ceph-mon[107832]: from='client.? 172.21.15.57:0/3184702918' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-01T19:38:43.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:42 smithi057 ceph-mon[107832]: from='client.? 172.21.15.57:0/3184702918' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-01T19:38:43.657 INFO:teuthology.orchestra.run.smithi057.stderr:Restarting the monitor... 2023-10-01T19:38:43.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:43 smithi057 systemd[1]: Stopping Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:38:43.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:43 smithi057 bash[108131]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon.a" found: no such container 2023-10-01T19:38:43.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-a[107809]: 2023-10-01T19:38:43.773+0000 7f90c53db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T19:38:43.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-a[107809]: 2023-10-01T19:38:43.773+0000 7f90c53db700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-01T19:38:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 bash[108144]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-a 2023-10-01T19:38:44.469 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 bash[108182]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon.a" found: no such container 2023-10-01T19:38:44.469 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-10-01T19:38:44.469 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 systemd[1]: Stopped Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:38:44.469 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 systemd[1]: Starting Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:38:44.604 INFO:teuthology.orchestra.run.smithi057.stderr:Setting mon public_network to 172.21.0.0/20 2023-10-01T19:38:44.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: set uid:gid to 167:167 (ceph:ceph) 2023-10-01T19:38:44.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable), process ceph-mon, pid 7 2023-10-01T19:38:44.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: pidfile_write: ignore empty --pid-file 2023-10-01T19:38:44.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 bash[108248]: 2f747ad4238019d8bf1f6fc007516c89b5742dbc10a044abde2bf8e08a15cec7 2023-10-01T19:38:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 systemd[1]: Started Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:38:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: load: jerasure load: lrc 2023-10-01T19:38:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: RocksDB version: 6.15.5 2023-10-01T19:38:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-10-01T19:38:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Compile date Apr 18 2022 2023-10-01T19:38:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: DB SUMMARY 2023-10-01T19:38:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: DB Session ID: X01ZAS4IPFX5ILDS7EQR 2023-10-01T19:38:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: CURRENT file: CURRENT 2023-10-01T19:38:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: IDENTITY file: IDENTITY 2023-10-01T19:38:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2023-10-01T19:38:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-01T19:38:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 80721 ; 2023-10-01T19:38:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.error_if_exists: 0 2023-10-01T19:38:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.create_if_missing: 0 2023-10-01T19:38:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.paranoid_checks: 1 2023-10-01T19:38:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-01T19:38:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.env: 0x5615fbaa7860 2023-10-01T19:38:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.fs: Posix File System 2023-10-01T19:38:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.info_log: 0x5615fd2c9dc0 2023-10-01T19:38:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_file_opening_threads: 16 2023-10-01T19:38:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.statistics: (nil) 2023-10-01T19:38:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.use_fsync: 0 2023-10-01T19:38:44.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_log_file_size: 0 2023-10-01T19:38:44.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-01T19:38:44.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-01T19:38:44.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.keep_log_file_num: 1000 2023-10-01T19:38:44.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.recycle_log_file_num: 0 2023-10-01T19:38:44.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.allow_fallocate: 1 2023-10-01T19:38:44.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.allow_mmap_reads: 0 2023-10-01T19:38:44.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.allow_mmap_writes: 0 2023-10-01T19:38:44.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.use_direct_reads: 0 2023-10-01T19:38:44.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-01T19:38:44.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.create_missing_column_families: 0 2023-10-01T19:38:44.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.db_log_dir: 2023-10-01T19:38:44.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2023-10-01T19:38:44.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-01T19:38:44.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-01T19:38:44.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-01T19:38:44.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-01T19:38:44.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-01T19:38:44.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-01T19:38:44.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.advise_random_on_open: 1 2023-10-01T19:38:44.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.db_write_buffer_size: 0 2023-10-01T19:38:44.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.write_buffer_manager: 0x5615fd3ba2a0 2023-10-01T19:38:44.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-01T19:38:44.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-10-01T19:38:44.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-01T19:38:44.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-01T19:38:44.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.rate_limiter: (nil) 2023-10-01T19:38:44.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-01T19:38:44.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.wal_recovery_mode: 2 2023-10-01T19:38:44.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.enable_thread_tracking: 0 2023-10-01T19:38:44.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.enable_pipelined_write: 0 2023-10-01T19:38:44.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.unordered_write: 0 2023-10-01T19:38:44.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-01T19:38:44.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-01T19:38:44.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-01T19:38:44.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-01T19:38:44.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.row_cache: None 2023-10-01T19:38:44.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.wal_filter: None 2023-10-01T19:38:44.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-01T19:38:44.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.allow_ingest_behind: 0 2023-10-01T19:38:44.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.preserve_deletes: 0 2023-10-01T19:38:44.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.two_write_queues: 0 2023-10-01T19:38:44.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.manual_wal_flush: 0 2023-10-01T19:38:44.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.atomic_flush: 0 2023-10-01T19:38:44.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-01T19:38:44.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-01T19:38:44.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-01T19:38:44.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.log_readahead_size: 0 2023-10-01T19:38:44.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-01T19:38:44.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.best_efforts_recovery: 0 2023-10-01T19:38:44.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-01T19:38:44.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-01T19:38:44.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.allow_data_in_errors: 0 2023-10-01T19:38:44.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.db_host_id: __hostname__ 2023-10-01T19:38:44.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_background_jobs: 2 2023-10-01T19:38:44.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_background_compactions: -1 2023-10-01T19:38:44.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_subcompactions: 1 2023-10-01T19:38:44.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-01T19:38:44.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-01T19:38:44.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-01T19:38:44.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_total_wal_size: 0 2023-10-01T19:38:44.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-01T19:38:44.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-01T19:38:44.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-01T19:38:44.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-01T19:38:44.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_open_files: -1 2023-10-01T19:38:44.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.bytes_per_sync: 0 2023-10-01T19:38:44.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-01T19:38:44.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-01T19:38:44.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_readahead_size: 0 2023-10-01T19:38:44.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_background_flushes: -1 2023-10-01T19:38:44.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Compression algorithms supported: 2023-10-01T19:38:44.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-01T19:38:44.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: kZSTD supported: 0 2023-10-01T19:38:44.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: kXpressCompression supported: 0 2023-10-01T19:38:44.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: kLZ4HCCompression supported: 1 2023-10-01T19:38:44.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: kLZ4Compression supported: 1 2023-10-01T19:38:44.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: kBZip2Compression supported: 0 2023-10-01T19:38:44.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: kZlibCompression supported: 1 2023-10-01T19:38:44.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: kSnappyCompression supported: 1 2023-10-01T19:38:44.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-01T19:38:44.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: [db/version_set.cc:4725] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2023-10-01T19:38:44.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: [db/column_family.cc:597] --------------- Options for column family [default]: 2023-10-01T19:38:44.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-01T19:38:44.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.merge_operator: 2023-10-01T19:38:44.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_filter: None 2023-10-01T19:38:44.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_filter_factory: None 2023-10-01T19:38:44.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.sst_partitioner_factory: None 2023-10-01T19:38:44.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-01T19:38:44.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-01T19:38:44.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5615fd295d00) 2023-10-01T19:38:44.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks: 1 2023-10-01T19:38:44.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-01T19:38:44.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-01T19:38:44.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_top_level_index_and_filter: 1 2023-10-01T19:38:44.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_type: 0 2023-10-01T19:38:44.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_index_type: 0 2023-10-01T19:38:44.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_shortening: 1 2023-10-01T19:38:44.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-01T19:38:44.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: hash_index_allow_collision: 1 2023-10-01T19:38:44.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: checksum: 1 2023-10-01T19:38:44.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: no_block_cache: 0 2023-10-01T19:38:44.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache: 0x5615fd300170 2023-10-01T19:38:44.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_name: BinnedLRUCache 2023-10-01T19:38:44.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_options: 2023-10-01T19:38:44.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: capacity : 536870912 2023-10-01T19:38:44.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_shard_bits : 4 2023-10-01T19:38:44.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: strict_capacity_limit : 0 2023-10-01T19:38:44.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: high_pri_pool_ratio: 0.000 2023-10-01T19:38:44.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_compressed: (nil) 2023-10-01T19:38:44.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: persistent_cache: (nil) 2023-10-01T19:38:44.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size: 4096 2023-10-01T19:38:44.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size_deviation: 10 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_restart_interval: 16 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_block_restart_interval: 1 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: metadata_block_size: 4096 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: partition_filters: 0 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: use_delta_encoding: 1 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: whole_key_filtering: 1 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: verify_compression: 0 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: read_amp_bytes_per_bit: 0 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: format_version: 4 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_index_compression: 1 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_align: 0 2023-10-01T19:38:44.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.write_buffer_size: 33554432 2023-10-01T19:38:44.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_write_buffer_number: 2 2023-10-01T19:38:44.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compression: NoCompression 2023-10-01T19:38:44.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.bottommost_compression: Disabled 2023-10-01T19:38:44.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.prefix_extractor: nullptr 2023-10-01T19:38:44.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-01T19:38:44.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.num_levels: 7 2023-10-01T19:38:44.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-01T19:38:44.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-01T19:38:44.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-01T19:38:44.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-01T19:38:44.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-01T19:38:44.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-01T19:38:44.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-01T19:38:44.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-01T19:38:44.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-01T19:38:44.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-01T19:38:44.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-01T19:38:44.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compression_opts.level: 32767 2023-10-01T19:38:44.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compression_opts.strategy: 0 2023-10-01T19:38:44.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-01T19:38:44.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-01T19:38:44.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-01T19:38:44.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compression_opts.enabled: false 2023-10-01T19:38:44.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-01T19:38:44.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-01T19:38:44.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-01T19:38:44.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.target_file_size_base: 67108864 2023-10-01T19:38:44.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-01T19:38:44.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-01T19:38:44.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-01T19:38:44.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-01T19:38:44.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-01T19:38:44.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-01T19:38:44.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-01T19:38:44.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-01T19:38:44.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-01T19:38:44.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-01T19:38:44.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-01T19:38:44.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-01T19:38:44.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-01T19:38:44.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.arena_block_size: 4194304 2023-10-01T19:38:44.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-01T19:38:44.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-01T19:38:44.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-10-01T19:38:44.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.disable_auto_compactions: 0 2023-10-01T19:38:44.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-01T19:38:44.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-01T19:38:44.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-01T19:38:44.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-01T19:38:44.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-01T19:38:44.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-01T19:38:44.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-01T19:38:44.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-01T19:38:44.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-01T19:38:44.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-01T19:38:44.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.table_properties_collectors: 2023-10-01T19:38:44.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.inplace_update_support: 0 2023-10-01T19:38:44.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-01T19:38:44.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-01T19:38:44.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-01T19:38:44.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-01T19:38:44.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.bloom_locality: 0 2023-10-01T19:38:44.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.max_successive_merges: 0 2023-10-01T19:38:44.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-01T19:38:44.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.paranoid_file_checks: 0 2023-10-01T19:38:44.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.force_consistency_checks: 1 2023-10-01T19:38:44.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.report_bg_io_stats: 0 2023-10-01T19:38:44.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.ttl: 2592000 2023-10-01T19:38:44.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-01T19:38:44.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.enable_blob_files: false 2023-10-01T19:38:44.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.min_blob_size: 0 2023-10-01T19:38:44.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.blob_file_size: 268435456 2023-10-01T19:38:44.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-01T19:38:44.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-01T19:38:44.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-01T19:38:44.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: [db/version_set.cc:4773] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-01T19:38:44.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: [db/version_set.cc:4782] Column family [default] (ID 0), log number is 5 2023-10-01T19:38:44.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: [db/version_set.cc:4083] Creating manifest 13 2023-10-01T19:38:44.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696189124608068, "job": 1, "event": "recovery_started", "wal_files": [10]} 2023-10-01T19:38:44.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: [db/db_impl/db_impl_open.cc:847] Recovering log #10 mode 2 2023-10-01T19:38:44.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: [table/block_based/filter_policy.cc:996] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-10-01T19:38:44.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696189124609111, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 77587, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 75850, "index_size": 225, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9840, "raw_average_key_size": 47, "raw_value_size": 70160, "raw_average_value_size": 340, "num_data_blocks": 10, "num_entries": 206, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1696189124, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "085fe613-ee20-4c38-a277-5a472245ad4d", "db_session_id": "X01ZAS4IPFX5ILDS7EQR"}} 2023-10-01T19:38:44.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: [db/version_set.cc:4083] Creating manifest 15 2023-10-01T19:38:44.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696189124610085, "job": 1, "event": "recovery_finished"} 2023-10-01T19:38:44.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: [file/delete_scheduler.cc:73] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-01T19:38:44.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: [db/db_impl/db_impl_open.cc:1701] SstFileManager instance 0x5615fd2e6a80 2023-10-01T19:38:44.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: DB pointer 0x5615fd2f6000 2023-10-01T19:38:44.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: [db/db_impl/db_impl.cc:902] ------- DUMPING STATS ------- 2023-10-01T19:38:44.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: rocksdb: [db/db_impl/db_impl.cc:903] 2023-10-01T19:38:44.944 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** DB Stats ** 2023-10-01T19:38:44.944 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T19:38:44.944 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-01T19:38:44.945 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-01T19:38:44.945 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T19:38:44.945 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-01T19:38:44.945 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-10-01T19:38:44.945 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T19:38:44.945 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T19:38:44.945 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T19:38:44.945 INFO:journalctl@ceph.mon.a.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-01T19:38:44.945 INFO:journalctl@ceph.mon.a.smithi057.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T19:38:44.945 INFO:journalctl@ceph.mon.a.smithi057.stdout: L0 2/0 77.53 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.8 0.00 0.00 1 0.001 0 0 2023-10-01T19:38:44.945 INFO:journalctl@ceph.mon.a.smithi057.stdout: Sum 2/0 77.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.8 0.00 0.00 1 0.001 0 0 2023-10-01T19:38:44.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.8 0.00 0.00 1 0.001 0 0 2023-10-01T19:38:44.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T19:38:44.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T19:38:44.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-01T19:38:44.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T19:38:44.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 112.8 0.00 0.00 1 0.001 0 0 2023-10-01T19:38:44.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T19:38:44.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-01T19:38:44.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-01T19:38:44.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-01T19:38:44.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-01T19:38:44.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-01T19:38:44.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative compaction: 0.00 GB write, 13.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T19:38:44.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval compaction: 0.00 GB write, 13.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T19:38:44.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-01T19:38:44.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T19:38:44.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-01T19:38:44.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T19:38:44.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T19:38:44.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-01T19:38:44.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T19:38:44.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: L0 2/0 77.53 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.8 0.00 0.00 1 0.001 0 0 2023-10-01T19:38:44.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: Sum 2/0 77.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.8 0.00 0.00 1 0.001 0 0 2023-10-01T19:38:44.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-10-01T19:38:44.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T19:38:44.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T19:38:44.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-01T19:38:44.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T19:38:44.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 112.8 0.00 0.00 1 0.001 0 0 2023-10-01T19:38:44.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T19:38:44.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-01T19:38:44.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-01T19:38:44.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-01T19:38:44.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-01T19:38:44.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-01T19:38:44.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative compaction: 0.00 GB write, 13.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: starting mon.a rank 0 at public addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] at bind addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: mon.a@-1(???) e1 preinit fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: mon.a@-1(???).mds e1 new map 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: mon.a@-1(???).mds e1 print_map 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: e1 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: legacy client fscid: -1 2023-10-01T19:38:44.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T19:38:44.950 INFO:journalctl@ceph.mon.a.smithi057.stdout: No filesystems configured 2023-10-01T19:38:44.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-01T19:38:44.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-01T19:38:44.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-01T19:38:44.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-01T19:38:44.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-01T19:38:44.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-01T19:38:44.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: monmap e1: 1 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0]} 2023-10-01T19:38:44.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: fsmap 2023-10-01T19:38:44.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: osdmap e1: 0 total, 0 up, 0 in 2023-10-01T19:38:44.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:44 smithi057 ceph-mon[108286]: mgrmap e1: no daemons active 2023-10-01T19:38:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:Wrote config to /etc/ceph/ceph.conf 2023-10-01T19:38:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-01T19:38:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:Creating mgr... 2023-10-01T19:38:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:Verifying port 9283 ... 2023-10-01T19:38:45.598 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: Failed to reset failed state of unit ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.y.service: Unit ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.y.service not loaded. 2023-10-01T19:38:45.605 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: Created symlink /etc/systemd/system/ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7.target.wants/ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.y.service → /etc/systemd/system/ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@.service. 2023-10-01T19:38:45.980 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:45 smithi057 systemd[1]: Starting Ceph mgr.y for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:38:46.149 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: disabled 2023-10-01T19:38:46.158 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: inactive 2023-10-01T19:38:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:firewalld.service is not enabled 2023-10-01T19:38:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:Not possible to enable service . firewalld.service is not available 2023-10-01T19:38:46.168 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: disabled 2023-10-01T19:38:46.176 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: inactive 2023-10-01T19:38:46.177 INFO:teuthology.orchestra.run.smithi057.stderr:firewalld.service is not enabled 2023-10-01T19:38:46.178 INFO:teuthology.orchestra.run.smithi057.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2023-10-01T19:38:46.178 INFO:teuthology.orchestra.run.smithi057.stderr:Waiting for mgr to start... 2023-10-01T19:38:46.178 INFO:teuthology.orchestra.run.smithi057.stderr:Waiting for mgr... 2023-10-01T19:38:46.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:46 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3173302710' entity='client.admin' 2023-10-01T19:38:46.325 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:46 smithi057 bash[108518]: d546d6fb718adb0cb62623f5af8a14af47e2f546ec89d8031965db09b5863c4b 2023-10-01T19:38:46.325 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:46 smithi057 systemd[1]: Started Ceph mgr.y for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:38:46.325 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:46.279+0000 7f78000b6000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T19:38:46.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:46.476+0000 7f78000b6000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T19:38:46.696 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:46.696 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: { 2023-10-01T19:38:46.696 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "fsid": "0f6d5dc0-6092-11ee-8db5-212e2dc638e7", 2023-10-01T19:38:46.696 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "health": { 2023-10-01T19:38:46.696 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2023-10-01T19:38:46.696 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "checks": {}, 2023-10-01T19:38:46.696 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "mutes": [] 2023-10-01T19:38:46.696 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:46.697 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "election_epoch": 5, 2023-10-01T19:38:46.697 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "quorum": [ 2023-10-01T19:38:46.697 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 0 2023-10-01T19:38:46.697 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ], 2023-10-01T19:38:46.697 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "quorum_names": [ 2023-10-01T19:38:46.697 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "a" 2023-10-01T19:38:46.697 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ], 2023-10-01T19:38:46.697 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "quorum_age": 2, 2023-10-01T19:38:46.697 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "monmap": { 2023-10-01T19:38:46.697 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:46.697 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "min_mon_release_name": "quincy", 2023-10-01T19:38:46.697 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_mons": 1 2023-10-01T19:38:46.698 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:46.698 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "osdmap": { 2023-10-01T19:38:46.698 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:46.698 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_osds": 0, 2023-10-01T19:38:46.698 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_up_osds": 0, 2023-10-01T19:38:46.698 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "osd_up_since": 0, 2023-10-01T19:38:46.698 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_in_osds": 0, 2023-10-01T19:38:46.701 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "osd_in_since": 0, 2023-10-01T19:38:46.701 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2023-10-01T19:38:46.701 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:46.701 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "pgmap": { 2023-10-01T19:38:46.701 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "pgs_by_state": [], 2023-10-01T19:38:46.702 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_pgs": 0, 2023-10-01T19:38:46.702 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_pools": 0, 2023-10-01T19:38:46.702 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_objects": 0, 2023-10-01T19:38:46.702 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "data_bytes": 0, 2023-10-01T19:38:46.702 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "bytes_used": 0, 2023-10-01T19:38:46.702 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "bytes_avail": 0, 2023-10-01T19:38:46.702 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "bytes_total": 0 2023-10-01T19:38:46.702 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:46.702 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "fsmap": { 2023-10-01T19:38:46.702 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:46.702 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "by_rank": [], 2023-10-01T19:38:46.702 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "up:standby": 0 2023-10-01T19:38:46.703 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:46.703 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "mgrmap": { 2023-10-01T19:38:46.703 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "available": false, 2023-10-01T19:38:46.703 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_standbys": 0, 2023-10-01T19:38:46.703 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "modules": [ 2023-10-01T19:38:46.703 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "iostat", 2023-10-01T19:38:46.703 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "nfs", 2023-10-01T19:38:46.703 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "restful" 2023-10-01T19:38:46.703 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ], 2023-10-01T19:38:46.703 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "services": {} 2023-10-01T19:38:46.703 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:46.703 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "servicemap": { 2023-10-01T19:38:46.704 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:46.704 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "modified": "2023-10-01T19:38:40.969563+0000", 2023-10-01T19:38:46.704 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "services": {} 2023-10-01T19:38:46.704 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:46.704 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "progress_events": {} 2023-10-01T19:38:46.704 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: } 2023-10-01T19:38:47.064 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:46.844+0000 7f78000b6000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T19:38:47.102 INFO:teuthology.orchestra.run.smithi057.stderr:mgr not available, waiting (1/15)... 2023-10-01T19:38:47.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:47 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/494171652' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-01T19:38:47.843 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:47 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:47.429+0000 7f78000b6000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T19:38:48.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:47 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:47.843+0000 7f78000b6000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T19:38:48.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:48 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:48.577+0000 7f78000b6000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T19:38:48.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:48 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:48.771+0000 7f78000b6000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T19:38:49.564 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:49.469+0000 7f78000b6000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T19:38:49.587 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:49.587 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: { 2023-10-01T19:38:49.587 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "fsid": "0f6d5dc0-6092-11ee-8db5-212e2dc638e7", 2023-10-01T19:38:49.587 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "health": { 2023-10-01T19:38:49.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2023-10-01T19:38:49.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "checks": {}, 2023-10-01T19:38:49.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "mutes": [] 2023-10-01T19:38:49.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:49.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "election_epoch": 5, 2023-10-01T19:38:49.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "quorum": [ 2023-10-01T19:38:49.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 0 2023-10-01T19:38:49.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ], 2023-10-01T19:38:49.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "quorum_names": [ 2023-10-01T19:38:49.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "a" 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ], 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "quorum_age": 4, 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "monmap": { 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "min_mon_release_name": "quincy", 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_mons": 1 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "osdmap": { 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_osds": 0, 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_up_osds": 0, 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "osd_up_since": 0, 2023-10-01T19:38:49.589 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_in_osds": 0, 2023-10-01T19:38:49.593 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "osd_in_since": 0, 2023-10-01T19:38:49.593 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2023-10-01T19:38:49.593 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:49.593 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "pgmap": { 2023-10-01T19:38:49.593 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "pgs_by_state": [], 2023-10-01T19:38:49.593 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_pgs": 0, 2023-10-01T19:38:49.593 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_pools": 0, 2023-10-01T19:38:49.593 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_objects": 0, 2023-10-01T19:38:49.593 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "data_bytes": 0, 2023-10-01T19:38:49.593 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "bytes_used": 0, 2023-10-01T19:38:49.594 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "bytes_avail": 0, 2023-10-01T19:38:49.594 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "bytes_total": 0 2023-10-01T19:38:49.594 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:49.594 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "fsmap": { 2023-10-01T19:38:49.594 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:49.594 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "by_rank": [], 2023-10-01T19:38:49.594 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "up:standby": 0 2023-10-01T19:38:49.594 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:49.594 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "mgrmap": { 2023-10-01T19:38:49.594 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "available": false, 2023-10-01T19:38:49.594 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_standbys": 0, 2023-10-01T19:38:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "modules": [ 2023-10-01T19:38:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "iostat", 2023-10-01T19:38:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "nfs", 2023-10-01T19:38:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "restful" 2023-10-01T19:38:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ], 2023-10-01T19:38:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "services": {} 2023-10-01T19:38:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "servicemap": { 2023-10-01T19:38:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "modified": "2023-10-01T19:38:40.969563+0000", 2023-10-01T19:38:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "services": {} 2023-10-01T19:38:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:49.596 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "progress_events": {} 2023-10-01T19:38:49.596 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: } 2023-10-01T19:38:49.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:49 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1390463073' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-01T19:38:49.873 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:49.564+0000 7f78000b6000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T19:38:49.873 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:49.668+0000 7f78000b6000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T19:38:49.873 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:49.772+0000 7f78000b6000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T19:38:49.936 INFO:teuthology.orchestra.run.smithi057.stderr:mgr not available, waiting (2/15)... 2023-10-01T19:38:50.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:49.873+0000 7f78000b6000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T19:38:50.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:49.977+0000 7f78000b6000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T19:38:50.625 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:50 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:50.318+0000 7f78000b6000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T19:38:50.625 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:50 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:50.423+0000 7f78000b6000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T19:38:50.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:50 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:50.625+0000 7f78000b6000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T19:38:50.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:50 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:50.718+0000 7f78000b6000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T19:38:51.831 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:51 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:51.516+0000 7f78000b6000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T19:38:51.831 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:51 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:51.680+0000 7f78000b6000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T19:38:52.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:51 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:51.830+0000 7f78000b6000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:38:52.549 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:52 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:52.279+0000 7f78000b6000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T19:38:52.549 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:52 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:52.432+0000 7f78000b6000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:38:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: { 2023-10-01T19:38:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "fsid": "0f6d5dc0-6092-11ee-8db5-212e2dc638e7", 2023-10-01T19:38:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "health": { 2023-10-01T19:38:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2023-10-01T19:38:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "checks": {}, 2023-10-01T19:38:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "mutes": [] 2023-10-01T19:38:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "election_epoch": 5, 2023-10-01T19:38:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "quorum": [ 2023-10-01T19:38:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 0 2023-10-01T19:38:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ], 2023-10-01T19:38:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "quorum_names": [ 2023-10-01T19:38:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "a" 2023-10-01T19:38:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ], 2023-10-01T19:38:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "quorum_age": 8, 2023-10-01T19:38:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "monmap": { 2023-10-01T19:38:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "min_mon_release_name": "quincy", 2023-10-01T19:38:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_mons": 1 2023-10-01T19:38:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "osdmap": { 2023-10-01T19:38:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_osds": 0, 2023-10-01T19:38:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_up_osds": 0, 2023-10-01T19:38:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "osd_up_since": 0, 2023-10-01T19:38:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_in_osds": 0, 2023-10-01T19:38:52.906 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "osd_in_since": 0, 2023-10-01T19:38:52.906 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2023-10-01T19:38:52.906 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:52.906 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "pgmap": { 2023-10-01T19:38:52.906 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "pgs_by_state": [], 2023-10-01T19:38:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_pgs": 0, 2023-10-01T19:38:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_pools": 0, 2023-10-01T19:38:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_objects": 0, 2023-10-01T19:38:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "data_bytes": 0, 2023-10-01T19:38:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "bytes_used": 0, 2023-10-01T19:38:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "bytes_avail": 0, 2023-10-01T19:38:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "bytes_total": 0 2023-10-01T19:38:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "fsmap": { 2023-10-01T19:38:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "by_rank": [], 2023-10-01T19:38:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "up:standby": 0 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "mgrmap": { 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "available": false, 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_standbys": 0, 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "modules": [ 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "iostat", 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "nfs", 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "restful" 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ], 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "services": {} 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "servicemap": { 2023-10-01T19:38:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "modified": "2023-10-01T19:38:40.969563+0000", 2023-10-01T19:38:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "services": {} 2023-10-01T19:38:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "progress_events": {} 2023-10-01T19:38:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: } 2023-10-01T19:38:53.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:52 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2451855033' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-01T19:38:53.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:52 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:52.894+0000 7f78000b6000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T19:38:53.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:53.027+0000 7f78000b6000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T19:38:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:mgr not available, waiting (3/15)... 2023-10-01T19:38:54.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: Activating manager daemon y 2023-10-01T19:38:54.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: mgrmap e2: y(active, starting, since 0.00484529s) 2023-10-01T19:38:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:38:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:38:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:38:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:38:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:38:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: Manager daemon y is now available 2023-10-01T19:38:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:38:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:38:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' 2023-10-01T19:38:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' 2023-10-01T19:38:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:53 smithi057 ceph-mon[108286]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' 2023-10-01T19:38:55.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:55 smithi057 ceph-mon[108286]: mgrmap e3: y(active, since 1.00834s) 2023-10-01T19:38:56.007 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:56.007 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: { 2023-10-01T19:38:56.007 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "fsid": "0f6d5dc0-6092-11ee-8db5-212e2dc638e7", 2023-10-01T19:38:56.007 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "health": { 2023-10-01T19:38:56.007 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2023-10-01T19:38:56.007 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "checks": {}, 2023-10-01T19:38:56.007 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "mutes": [] 2023-10-01T19:38:56.007 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "election_epoch": 5, 2023-10-01T19:38:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "quorum": [ 2023-10-01T19:38:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 0 2023-10-01T19:38:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ], 2023-10-01T19:38:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "quorum_names": [ 2023-10-01T19:38:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "a" 2023-10-01T19:38:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ], 2023-10-01T19:38:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "quorum_age": 11, 2023-10-01T19:38:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "monmap": { 2023-10-01T19:38:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "min_mon_release_name": "quincy", 2023-10-01T19:38:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_mons": 1 2023-10-01T19:38:56.009 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:56.009 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "osdmap": { 2023-10-01T19:38:56.009 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:56.009 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_osds": 0, 2023-10-01T19:38:56.009 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_up_osds": 0, 2023-10-01T19:38:56.009 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "osd_up_since": 0, 2023-10-01T19:38:56.009 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_in_osds": 0, 2023-10-01T19:38:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "osd_in_since": 0, 2023-10-01T19:38:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2023-10-01T19:38:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "pgmap": { 2023-10-01T19:38:56.013 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "pgs_by_state": [], 2023-10-01T19:38:56.013 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_pgs": 0, 2023-10-01T19:38:56.013 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_pools": 0, 2023-10-01T19:38:56.013 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_objects": 0, 2023-10-01T19:38:56.013 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "data_bytes": 0, 2023-10-01T19:38:56.013 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "bytes_used": 0, 2023-10-01T19:38:56.013 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "bytes_avail": 0, 2023-10-01T19:38:56.013 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "bytes_total": 0 2023-10-01T19:38:56.013 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:56.013 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "fsmap": { 2023-10-01T19:38:56.013 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:56.013 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "by_rank": [], 2023-10-01T19:38:56.014 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "up:standby": 0 2023-10-01T19:38:56.014 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:56.014 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "mgrmap": { 2023-10-01T19:38:56.014 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "available": true, 2023-10-01T19:38:56.014 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_standbys": 0, 2023-10-01T19:38:56.014 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "modules": [ 2023-10-01T19:38:56.014 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "iostat", 2023-10-01T19:38:56.014 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "nfs", 2023-10-01T19:38:56.014 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "restful" 2023-10-01T19:38:56.014 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ], 2023-10-01T19:38:56.014 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "services": {} 2023-10-01T19:38:56.014 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "servicemap": { 2023-10-01T19:38:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 1, 2023-10-01T19:38:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "modified": "2023-10-01T19:38:40.969563+0000", 2023-10-01T19:38:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "services": {} 2023-10-01T19:38:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: }, 2023-10-01T19:38:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "progress_events": {} 2023-10-01T19:38:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: } 2023-10-01T19:38:56.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:56 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/590837300' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-01T19:38:56.359 INFO:teuthology.orchestra.run.smithi057.stderr:mgr is available 2023-10-01T19:38:56.928 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:56.928 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: [global] 2023-10-01T19:38:56.928 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: fsid = 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:38:56.928 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2023-10-01T19:38:56.928 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2023-10-01T19:38:56.928 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2023-10-01T19:38:56.928 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2023-10-01T19:38:56.929 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:56.929 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: [mgr] 2023-10-01T19:38:56.929 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2023-10-01T19:38:56.929 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 2023-10-01T19:38:56.929 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: [osd] 2023-10-01T19:38:56.929 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2023-10-01T19:38:56.929 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2023-10-01T19:38:57.294 INFO:teuthology.orchestra.run.smithi057.stderr:Enabling cephadm module... 2023-10-01T19:38:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:57 smithi057 ceph-mon[108286]: mgrmap e4: y(active, since 3s) 2023-10-01T19:38:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:57 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1908657503' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-01T19:38:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:57 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1908657503' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-01T19:38:58.612 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:58 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1136967829' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-01T19:38:58.613 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:58 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ignoring --setuser ceph since I am not root 2023-10-01T19:38:58.613 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:58 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ignoring --setgroup ceph since I am not root 2023-10-01T19:38:58.613 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:58 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:58.422+0000 7fd01fd41000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T19:38:58.613 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:58 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:58.612+0000 7fd01fd41000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T19:38:59.261 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:58 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:58.950+0000 7fd01fd41000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T19:38:59.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:59 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1136967829' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-01T19:38:59.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:38:59 smithi057 ceph-mon[108286]: mgrmap e5: y(active, since 5s) 2023-10-01T19:38:59.872 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:59 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:59.516+0000 7fd01fd41000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T19:39:00.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:38:59 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:38:59.930+0000 7fd01fd41000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T19:39:00.889 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:00.681+0000 7fd01fd41000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T19:39:01.587 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: { 2023-10-01T19:39:01.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 5, 2023-10-01T19:39:01.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "available": true, 2023-10-01T19:39:01.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "active_name": "y", 2023-10-01T19:39:01.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_standby": 0 2023-10-01T19:39:01.588 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: } 2023-10-01T19:39:01.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:01 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2384895184' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-01T19:39:01.896 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:00.889+0000 7fd01fd41000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T19:39:01.896 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:01 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:01.599+0000 7fd01fd41000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T19:39:01.896 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:01 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:01.695+0000 7fd01fd41000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T19:39:01.896 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:01 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:01.792+0000 7fd01fd41000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T19:39:01.896 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:01 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:01.896+0000 7fd01fd41000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T19:39:02.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:01 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:01.998+0000 7fd01fd41000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T19:39:02.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:02 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:02.101+0000 7fd01fd41000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T19:39:02.757 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:02 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:02.445+0000 7fd01fd41000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T19:39:02.757 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:02 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:02.551+0000 7fd01fd41000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T19:39:03.054 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:02 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:02.757+0000 7fd01fd41000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T19:39:03.054 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:02 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:02.851+0000 7fd01fd41000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T19:39:03.476 INFO:teuthology.orchestra.run.smithi057.stderr:Waiting for the mgr to restart... 2023-10-01T19:39:03.477 INFO:teuthology.orchestra.run.smithi057.stderr:Waiting for mgr epoch 5... 2023-10-01T19:39:03.823 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:03.657+0000 7fd01fd41000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T19:39:04.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:03.823+0000 7fd01fd41000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T19:39:04.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:03.977+0000 7fd01fd41000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:39:04.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:04 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:04.436+0000 7fd01fd41000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T19:39:04.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:04 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:04.591+0000 7fd01fd41000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:39:05.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:05 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:05.039+0000 7fd01fd41000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T19:39:05.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:05 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:05.170+0000 7fd01fd41000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T19:39:05.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:05 smithi057 ceph-mon[108286]: Active manager daemon y restarted 2023-10-01T19:39:05.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:05 smithi057 ceph-mon[108286]: Activating manager daemon y 2023-10-01T19:39:05.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:05 smithi057 ceph-mon[108286]: osdmap e2: 0 total, 0 up, 0 in 2023-10-01T19:39:06.656 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: { 2023-10-01T19:39:06.656 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2023-10-01T19:39:06.656 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "initialized": true 2023-10-01T19:39:06.657 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: } 2023-10-01T19:39:06.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: mgrmap e6: y(active, starting, since 0.453981s) 2023-10-01T19:39:06.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:39:06.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:39:06.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:39:06.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:39:06.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:39:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: Manager daemon y is now available 2023-10-01T19:39:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:39:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:39:07.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:06 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:39:06] ENGINE Bus STARTING 2023-10-01T19:39:07.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:07 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:39:07] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:39:07.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:07 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:39:07] ENGINE Bus STARTED 2023-10-01T19:39:07.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:07 smithi057 ceph-mon[108286]: mgrmap e7: y(active, since 1.46582s) 2023-10-01T19:39:07.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:07 smithi057 ceph-mon[108286]: [01/Oct/2023:19:39:06] ENGINE Bus STARTING 2023-10-01T19:39:07.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:07 smithi057 ceph-mon[108286]: [01/Oct/2023:19:39:07] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:39:07.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:07 smithi057 ceph-mon[108286]: [01/Oct/2023:19:39:07] ENGINE Bus STARTED 2023-10-01T19:39:07.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:07 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:07.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:07 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:08.610 INFO:teuthology.orchestra.run.smithi057.stderr:mgr epoch 5 is available 2023-10-01T19:39:08.610 INFO:teuthology.orchestra.run.smithi057.stderr:Setting orchestrator backend to cephadm... 2023-10-01T19:39:09.385 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:09 smithi057 ceph-mon[108286]: mgrmap e8: y(active, since 2s) 2023-10-01T19:39:10.879 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: value unchanged 2023-10-01T19:39:11.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:10 smithi057 ceph-mon[108286]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:11.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:10 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:11.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:10 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:11.257 INFO:teuthology.orchestra.run.smithi057.stderr:Generating ssh key... 2023-10-01T19:39:12.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:11 smithi057 ceph-mon[108286]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:12.361 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: Generating public/private rsa key pair. 2023-10-01T19:39:12.361 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: Your identification has been saved in /tmp/tmpj05tunqt/key. 2023-10-01T19:39:12.361 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: Your public key has been saved in /tmp/tmpj05tunqt/key.pub. 2023-10-01T19:39:12.361 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: The key fingerprint is: 2023-10-01T19:39:12.361 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: SHA256:/t/APX3Y9EKcGBQM9ezz8u6tLHXVusWpuwtmQQpAFgE ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:39:12.361 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: The key's randomart image is: 2023-10-01T19:39:12.361 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: +---[RSA 3072]----+ 2023-10-01T19:39:12.361 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: | Eo*o .++. | 2023-10-01T19:39:12.361 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: | . . ..o | 2023-10-01T19:39:12.361 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: | . .. o .| 2023-10-01T19:39:12.362 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: | . o = .o| 2023-10-01T19:39:12.362 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: | S. .. *o+| 2023-10-01T19:39:12.362 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: | . ..ooO=| 2023-10-01T19:39:12.362 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: | . +o.B+*| 2023-10-01T19:39:12.362 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: | .o o=.=o| 2023-10-01T19:39:12.362 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: | ...=B+=| 2023-10-01T19:39:12.362 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: +----[SHA256]-----+ 2023-10-01T19:39:12.942 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQmo8u9RTVob76KOgLcqjxAIm3BD1BwC4+uNkit+lGWjP4zWLOhN5bo/b89btsOgKybH8eIwvpesasqvkozIZGDUs/jHnvPG7xj/Nfpg26ZVPq5ygTF5GKRDrXmKUTXPtjyDbLkCxjdCb6H1ocW7T5rToC3GQ6V0+rx53SG7Sr7Q6tnprW3e7nP9Ew+n9dv2Zd4FfMrUbBiHmP+WMtujIJBbcGX6HNAa3wHQe5LL8T+iEM+qFnW5HbKJAY1YXubO8L32SGabHJX1dVkX4sY2bMGLjVrjTDa4bC9uhLbRZcvO4ORnbW86+vMFCooG5gleNsNMQMx6CtGt+ZFr/Ij4W65HdEOwPb6WtA7emHgRSm2CKOxVBD4oqA72Yx+L/GudEP8Y40LwSoNZg3a7y+eUk6sqMv7Q4BlJ6+GtuNmhTVYEGHE7kCDxxufPTwtOpfD6EVf4x5fxYgd7KO/i9cO6u78D7nibEquJvnLxvnQc9rYLDv8MPgVeedLo/q5L2Lmhc= ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:39:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:13 smithi057 ceph-mon[108286]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:13 smithi057 ceph-mon[108286]: Generating ssh key... 2023-10-01T19:39:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:13 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:13 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:13.299 INFO:teuthology.orchestra.run.smithi057.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-01T19:39:13.300 INFO:teuthology.orchestra.run.smithi057.stderr:Adding key to root@localhost authorized_keys... 2023-10-01T19:39:13.301 INFO:teuthology.orchestra.run.smithi057.stderr:Adding host smithi057... 2023-10-01T19:39:14.156 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:14 smithi057 ceph-mon[108286]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:14.791 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: Added host 'smithi057' with addr '172.21.15.57' 2023-10-01T19:39:15.140 INFO:teuthology.orchestra.run.smithi057.stderr:Deploying unmanaged mon service... 2023-10-01T19:39:15.306 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:15 smithi057 ceph-mon[108286]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:15.306 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:15 smithi057 ceph-mon[108286]: Deploying cephadm binary to smithi057 2023-10-01T19:39:15.306 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:15 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:15.306 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:15 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:15.712 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: Scheduled mon update... 2023-10-01T19:39:16.149 INFO:teuthology.orchestra.run.smithi057.stderr:Deploying unmanaged mgr service... 2023-10-01T19:39:16.302 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:16 smithi057 ceph-mon[108286]: Added host smithi057 2023-10-01T19:39:16.302 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:16 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:16.767 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: Scheduled mgr update... 2023-10-01T19:39:17.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:17 smithi057 ceph-mon[108286]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:17.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:17 smithi057 ceph-mon[108286]: Saving service mon spec with placement count:5 2023-10-01T19:39:17.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:17 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:18 smithi057 ceph-mon[108286]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:18 smithi057 ceph-mon[108286]: Saving service mgr spec with placement count:2 2023-10-01T19:39:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:18 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:18 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:18.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:18 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3571988148' entity='client.admin' 2023-10-01T19:39:19.373 INFO:teuthology.orchestra.run.smithi057.stderr:Enabling the dashboard module... 2023-10-01T19:39:19.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:19 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:19.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:19 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:19.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:19 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:39:19.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:19 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:19.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:19 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:39:19.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:19 smithi057 ceph-mon[108286]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]': finished 2023-10-01T19:39:19.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:19 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/265524055' entity='client.admin' 2023-10-01T19:39:20.910 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:20 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3249544287' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-01T19:39:20.910 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:20 smithi057 ceph-mon[108286]: Deploying daemon agent.smithi057 on smithi057 2023-10-01T19:39:20.910 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:20 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ignoring --setuser ceph since I am not root 2023-10-01T19:39:20.910 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:20 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ignoring --setgroup ceph since I am not root 2023-10-01T19:39:20.911 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:20 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:20.816+0000 7f55f6148000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T19:39:21.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:21 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:21.008+0000 7f55f6148000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T19:39:21.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: { 2023-10-01T19:39:21.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "epoch": 9, 2023-10-01T19:39:21.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "available": true, 2023-10-01T19:39:21.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "active_name": "y", 2023-10-01T19:39:21.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "num_standby": 0 2023-10-01T19:39:21.595 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: } 2023-10-01T19:39:21.625 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:21 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:21.357+0000 7f55f6148000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T19:39:21.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:21 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3249544287' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-01T19:39:21.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:21 smithi057 ceph-mon[108286]: mgrmap e9: y(active, since 15s) 2023-10-01T19:39:21.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:21 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3538706024' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-01T19:39:21.931 INFO:teuthology.orchestra.run.smithi057.stderr:Waiting for the mgr to restart... 2023-10-01T19:39:21.931 INFO:teuthology.orchestra.run.smithi057.stderr:Waiting for mgr epoch 9... 2023-10-01T19:39:22.373 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:21 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:21.935+0000 7f55f6148000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T19:39:22.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:22.372+0000 7f55f6148000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T19:39:23.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:23.104+0000 7f55f6148000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T19:39:23.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:23.297+0000 7f55f6148000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T19:39:24.281 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:23.986+0000 7f55f6148000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T19:39:24.281 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:24.079+0000 7f55f6148000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T19:39:24.281 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:24.176+0000 7f55f6148000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T19:39:24.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:24.279+0000 7f55f6148000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T19:39:24.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:24.381+0000 7f55f6148000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T19:39:24.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:24.484+0000 7f55f6148000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T19:39:25.142 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:24.830+0000 7f55f6148000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T19:39:25.143 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:24.936+0000 7f55f6148000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T19:39:25.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:25 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:25.141+0000 7f55f6148000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T19:39:25.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:25 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:25.236+0000 7f55f6148000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T19:39:26.357 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:26.038+0000 7f55f6148000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T19:39:26.357 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:26.205+0000 7f55f6148000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T19:39:26.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:26.355+0000 7f55f6148000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:39:27.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:26.801+0000 7f55f6148000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T19:39:27.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:26.954+0000 7f55f6148000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:39:27.666 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:27 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:27.397+0000 7f55f6148000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T19:39:27.666 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:27 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:39:27.525+0000 7f55f6148000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T19:39:28.608 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: { 2023-10-01T19:39:28.608 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "mgrmap_epoch": 11, 2023-10-01T19:39:28.608 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: "initialized": true 2023-10-01T19:39:28.608 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: } 2023-10-01T19:39:28.770 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:39:28] ENGINE Bus STARTING 2023-10-01T19:39:28.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: Active manager daemon y restarted 2023-10-01T19:39:28.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: Activating manager daemon y 2023-10-01T19:39:28.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: osdmap e3: 0 total, 0 up, 0 in 2023-10-01T19:39:28.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: mgrmap e10: y(active, starting, since 0.0531447s) 2023-10-01T19:39:28.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:39:28.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:39:28.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:39:28.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:39:28.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:39:28.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: Manager daemon y is now available 2023-10-01T19:39:28.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:28.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:28.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:28.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:39:28.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:39:28.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:28.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:39:28.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:28.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:39:28.963 INFO:teuthology.orchestra.run.smithi057.stderr:mgr epoch 9 is available 2023-10-01T19:39:28.963 INFO:teuthology.orchestra.run.smithi057.stderr:Generating a dashboard self-signed certificate... 2023-10-01T19:39:29.078 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:39:28] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:39:29.078 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:28 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:39:28] ENGINE Bus STARTED 2023-10-01T19:39:29.647 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: Self-signed certificate created 2023-10-01T19:39:29.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:29 smithi057 ceph-mon[108286]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-01T19:39:29.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:29 smithi057 ceph-mon[108286]: mgrmap e11: y(active, since 1.05579s) 2023-10-01T19:39:29.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:29 smithi057 ceph-mon[108286]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-01T19:39:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:29 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:Creating initial admin user... 2023-10-01T19:39:30.900 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$mE9rcXGEOynx5Usg4g9iyuWqUpTzAdcdH5R7/OX8pWuawkf0t.DK2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1696189170, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-01T19:39:30.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:30 smithi057 ceph-mon[108286]: [01/Oct/2023:19:39:28] ENGINE Bus STARTING 2023-10-01T19:39:30.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:30 smithi057 ceph-mon[108286]: [01/Oct/2023:19:39:28] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:39:30.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:30 smithi057 ceph-mon[108286]: [01/Oct/2023:19:39:28] ENGINE Bus STARTED 2023-10-01T19:39:30.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:30 smithi057 ceph-mon[108286]: Deploying daemon agent.smithi057 on smithi057 2023-10-01T19:39:30.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:30 smithi057 ceph-mon[108286]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:30.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:30 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:30.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:30 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:30.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:30 smithi057 ceph-mon[108286]: mgrmap e12: y(active, since 2s) 2023-10-01T19:39:31.263 INFO:teuthology.orchestra.run.smithi057.stderr:Fetching dashboard port number... 2023-10-01T19:39:31.854 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: 8443 2023-10-01T19:39:31.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:31 smithi057 ceph-mon[108286]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:31.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:31 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:32.266 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: disabled 2023-10-01T19:39:32.275 INFO:teuthology.orchestra.run.smithi057.stderr:systemctl: inactive 2023-10-01T19:39:32.276 INFO:teuthology.orchestra.run.smithi057.stderr:firewalld.service is not enabled 2023-10-01T19:39:32.276 INFO:teuthology.orchestra.run.smithi057.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-01T19:39:32.279 INFO:teuthology.orchestra.run.smithi057.stderr:Ceph Dashboard is now available at: 2023-10-01T19:39:32.279 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-01T19:39:32.279 INFO:teuthology.orchestra.run.smithi057.stderr: URL: https://smithi057.front.sepia.ceph.com:8443/ 2023-10-01T19:39:32.280 INFO:teuthology.orchestra.run.smithi057.stderr: User: admin 2023-10-01T19:39:32.280 INFO:teuthology.orchestra.run.smithi057.stderr: Password: hqq3z5338o 2023-10-01T19:39:32.280 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-01T19:39:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:Enabling autotune for osd_memory_target 2023-10-01T19:39:32.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:32 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/413849774' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-01T19:39:33.952 INFO:teuthology.orchestra.run.smithi057.stderr:/usr/bin/ceph: set mgr/dashboard/cluster/status 2023-10-01T19:39:34.287 INFO:teuthology.orchestra.run.smithi057.stderr:You can access the Ceph CLI with: 2023-10-01T19:39:34.287 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-01T19:39:34.288 INFO:teuthology.orchestra.run.smithi057.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-01T19:39:34.288 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-01T19:39:34.288 INFO:teuthology.orchestra.run.smithi057.stderr:Please consider enabling telemetry to help improve Ceph: 2023-10-01T19:39:34.288 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-01T19:39:34.288 INFO:teuthology.orchestra.run.smithi057.stderr: ceph telemetry on 2023-10-01T19:39:34.288 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-01T19:39:34.288 INFO:teuthology.orchestra.run.smithi057.stderr:For more information see: 2023-10-01T19:39:34.288 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-01T19:39:34.288 INFO:teuthology.orchestra.run.smithi057.stderr: https://docs.ceph.com/docs/master/mgr/telemetry/ 2023-10-01T19:39:34.288 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-01T19:39:34.288 INFO:teuthology.orchestra.run.smithi057.stderr:Bootstrap complete. 2023-10-01T19:39:34.335 INFO:tasks.cephadm:Fetching config... 2023-10-01T19:39:34.335 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-01T19:39:34.335 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-01T19:39:34.368 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-01T19:39:34.368 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-01T19:39:34.369 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-01T19:39:34.429 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-01T19:39:34.429 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-01T19:39:34.429 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-01T19:39:34.504 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-01T19:39:34.504 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-01T19:39:34.504 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-01T19:39:34.563 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-01T19:39:34.563 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQmo8u9RTVob76KOgLcqjxAIm3BD1BwC4+uNkit+lGWjP4zWLOhN5bo/b89btsOgKybH8eIwvpesasqvkozIZGDUs/jHnvPG7xj/Nfpg26ZVPq5ygTF5GKRDrXmKUTXPtjyDbLkCxjdCb6H1ocW7T5rToC3GQ6V0+rx53SG7Sr7Q6tnprW3e7nP9Ew+n9dv2Zd4FfMrUbBiHmP+WMtujIJBbcGX6HNAa3wHQe5LL8T+iEM+qFnW5HbKJAY1YXubO8L32SGabHJX1dVkX4sY2bMGLjVrjTDa4bC9uhLbRZcvO4ORnbW86+vMFCooG5gleNsNMQMx6CtGt+ZFr/Ij4W65HdEOwPb6WtA7emHgRSm2CKOxVBD4oqA72Yx+L/GudEP8Y40LwSoNZg3a7y+eUk6sqMv7Q4BlJ6+GtuNmhTVYEGHE7kCDxxufPTwtOpfD6EVf4x5fxYgd7KO/i9cO6u78D7nibEquJvnLxvnQc9rYLDv8MPgVeedLo/q5L2Lmhc= ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-01T19:39:34.651 INFO:teuthology.orchestra.run.smithi057.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQmo8u9RTVob76KOgLcqjxAIm3BD1BwC4+uNkit+lGWjP4zWLOhN5bo/b89btsOgKybH8eIwvpesasqvkozIZGDUs/jHnvPG7xj/Nfpg26ZVPq5ygTF5GKRDrXmKUTXPtjyDbLkCxjdCb6H1ocW7T5rToC3GQ6V0+rx53SG7Sr7Q6tnprW3e7nP9Ew+n9dv2Zd4FfMrUbBiHmP+WMtujIJBbcGX6HNAa3wHQe5LL8T+iEM+qFnW5HbKJAY1YXubO8L32SGabHJX1dVkX4sY2bMGLjVrjTDa4bC9uhLbRZcvO4ORnbW86+vMFCooG5gleNsNMQMx6CtGt+ZFr/Ij4W65HdEOwPb6WtA7emHgRSm2CKOxVBD4oqA72Yx+L/GudEP8Y40LwSoNZg3a7y+eUk6sqMv7Q4BlJ6+GtuNmhTVYEGHE7kCDxxufPTwtOpfD6EVf4x5fxYgd7KO/i9cO6u78D7nibEquJvnLxvnQc9rYLDv8MPgVeedLo/q5L2Lmhc= ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:39:34.669 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQmo8u9RTVob76KOgLcqjxAIm3BD1BwC4+uNkit+lGWjP4zWLOhN5bo/b89btsOgKybH8eIwvpesasqvkozIZGDUs/jHnvPG7xj/Nfpg26ZVPq5ygTF5GKRDrXmKUTXPtjyDbLkCxjdCb6H1ocW7T5rToC3GQ6V0+rx53SG7Sr7Q6tnprW3e7nP9Ew+n9dv2Zd4FfMrUbBiHmP+WMtujIJBbcGX6HNAa3wHQe5LL8T+iEM+qFnW5HbKJAY1YXubO8L32SGabHJX1dVkX4sY2bMGLjVrjTDa4bC9uhLbRZcvO4ORnbW86+vMFCooG5gleNsNMQMx6CtGt+ZFr/Ij4W65HdEOwPb6WtA7emHgRSm2CKOxVBD4oqA72Yx+L/GudEP8Y40LwSoNZg3a7y+eUk6sqMv7Q4BlJ6+GtuNmhTVYEGHE7kCDxxufPTwtOpfD6EVf4x5fxYgd7KO/i9cO6u78D7nibEquJvnLxvnQc9rYLDv8MPgVeedLo/q5L2Lmhc= ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-01T19:39:34.714 INFO:teuthology.orchestra.run.smithi087.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQmo8u9RTVob76KOgLcqjxAIm3BD1BwC4+uNkit+lGWjP4zWLOhN5bo/b89btsOgKybH8eIwvpesasqvkozIZGDUs/jHnvPG7xj/Nfpg26ZVPq5ygTF5GKRDrXmKUTXPtjyDbLkCxjdCb6H1ocW7T5rToC3GQ6V0+rx53SG7Sr7Q6tnprW3e7nP9Ew+n9dv2Zd4FfMrUbBiHmP+WMtujIJBbcGX6HNAa3wHQe5LL8T+iEM+qFnW5HbKJAY1YXubO8L32SGabHJX1dVkX4sY2bMGLjVrjTDa4bC9uhLbRZcvO4ORnbW86+vMFCooG5gleNsNMQMx6CtGt+ZFr/Ij4W65HdEOwPb6WtA7emHgRSm2CKOxVBD4oqA72Yx+L/GudEP8Y40LwSoNZg3a7y+eUk6sqMv7Q4BlJ6+GtuNmhTVYEGHE7kCDxxufPTwtOpfD6EVf4x5fxYgd7KO/i9cO6u78D7nibEquJvnLxvnQc9rYLDv8MPgVeedLo/q5L2Lmhc= ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:39:34.726 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-01T19:39:34.899 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:34 smithi057 ceph-mon[108286]: mgrmap e13: y(active, since 6s) 2023-10-01T19:39:34.899 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:34 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3208523705' entity='client.admin' 2023-10-01T19:39:35.661 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:35 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:39:35] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:39:35.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:35 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:35.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:35 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:35.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:35 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:35.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:35 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:35.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:35 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:35.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:35 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:37.580 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-01T19:39:37.581 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-01T19:39:37.886 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:37 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:39:37] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:39:38.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:38 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:39:38] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:39:38.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:38 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2830144859' entity='client.admin' 2023-10-01T19:39:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:39.469 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:39 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:39.469 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:39 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:39.684 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi087 2023-10-01T19:39:39.684 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-10-01T19:39:39.684 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/etc/ceph/ceph.conf 2023-10-01T19:39:39.707 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-10-01T19:39:39.707 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:39:39.768 INFO:tasks.cephadm:Adding host smithi087 to orchestrator... 2023-10-01T19:39:39.768 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi087 2023-10-01T19:39:40.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:40 smithi057 ceph-mon[108286]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:40.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:40 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:40.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:40 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:40.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:40 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:40.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:40 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:39:40.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:40 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:39:40.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:40 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:39:40.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:40 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:40.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:40 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:42.096 INFO:teuthology.orchestra.run.smithi057.stdout:Added host 'smithi087' with addr '172.21.15.87' 2023-10-01T19:39:42.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:41 smithi057 ceph-mon[108286]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi087", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:42.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:41 smithi057 ceph-mon[108286]: Deploying cephadm binary to smithi087 2023-10-01T19:39:42.464 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-01T19:39:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:43 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:43 smithi057 ceph-mon[108286]: Added host smithi087 2023-10-01T19:39:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:43 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:43 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:43 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:39:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:43 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:43 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:43 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:43 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:39:43.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:43 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:43.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:43 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:39:43.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:43 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]': finished 2023-10-01T19:39:43.916 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:39:43.916 INFO:teuthology.orchestra.run.smithi057.stdout:[{"addr": "172.21.15.57", "hostname": "smithi057", "labels": [], "status": ""}, {"addr": "172.21.15.87", "hostname": "smithi087", "labels": [], "status": ""}] 2023-10-01T19:39:44.223 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:44 smithi057 ceph-mon[108286]: Deploying daemon agent.smithi087 on smithi087 2023-10-01T19:39:44.305 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-01T19:39:44.305 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd crush tunables default 2023-10-01T19:39:44.795 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:39:44] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:39:45.110 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:39:44] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:39:45.374 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-01T19:39:45.374 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:45.374 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:45.374 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:45.374 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:39:45.374 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:39:45.374 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:39:45.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:45.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:45.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:45.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:45.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:45.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:39:45.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:45.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:45.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:45 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:46.125 INFO:teuthology.orchestra.run.smithi057.stderr:adjusted tunables profile to default 2023-10-01T19:39:46.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:46 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1026934141' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-01T19:39:46.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:39:46] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:39:46.505 INFO:tasks.cephadm:Adding mon.a on smithi057 2023-10-01T19:39:46.505 INFO:tasks.cephadm:Adding mon.c on smithi057 2023-10-01T19:39:46.505 INFO:tasks.cephadm:Adding mon.b on smithi087 2023-10-01T19:39:46.505 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch apply mon '3;smithi057:172.21.15.57=a;smithi057:[v2:172.21.15.57:3301,v1:172.21.15.57:6790]=c;smithi087:172.21.15.87=b' 2023-10-01T19:39:47.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:47 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1026934141' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-01T19:39:47.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:47 smithi057 ceph-mon[108286]: osdmap e4: 0 total, 0 up, 0 in 2023-10-01T19:39:47.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:47 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:47.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:47 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:47.832 INFO:teuthology.orchestra.run.smithi087.stdout:Scheduled mon update... 2023-10-01T19:39:48.196 DEBUG:teuthology.orchestra.run.smithi057:mon.c> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.c.service 2023-10-01T19:39:48.198 DEBUG:teuthology.orchestra.run.smithi087:mon.b> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.b.service 2023-10-01T19:39:48.200 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-01T19:39:48.200 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-01T19:39:48.230 INFO:journalctl@ceph.mon.b.smithi087.stdout:-- Logs begin at Sun 2023-10-01 19:27:12 UTC. -- 2023-10-01T19:39:48.231 INFO:journalctl@ceph.mon.c.smithi057.stdout:-- Logs begin at Sun 2023-10-01 19:27:22 UTC. -- 2023-10-01T19:39:48.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:48 smithi057 ceph-mon[108286]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:39:48.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:48.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:48.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:48.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:48.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:39:48.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:48.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:39:48.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:49.566 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:39:49.566 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":1,"fsid":"0f6d5dc0-6092-11ee-8db5-212e2dc638e7","modified":"2023-10-01T19:38:38.894526Z","created":"2023-10-01T19:38:38.894526Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-01T19:39:49.568 INFO:teuthology.orchestra.run.smithi087.stderr:dumped monmap epoch 1 2023-10-01T19:39:49.789 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:49 smithi057 systemd[1]: Starting Ceph mon.c for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:39:49.789 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:49 smithi057 ceph-mon[108286]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi057:[v2:172.21.15.57:3301,v1:172.21.15.57:6790]=c;smithi087:172.21.15.87=b", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:49.789 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:49 smithi057 ceph-mon[108286]: Saving service mon spec with placement smithi057:172.21.15.57=a;smithi057:[v2:172.21.15.57:3301,v1:172.21.15.57:6790]=c;smithi087:172.21.15.87=b;count:3 2023-10-01T19:39:49.789 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:49 smithi057 ceph-mon[108286]: Deploying daemon mon.c on smithi057 2023-10-01T19:39:49.789 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:49 smithi057 ceph-mon[108286]: from='client.? 172.21.15.87:0/3603447744' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: set uid:gid to 167:167 (ceph:ceph) 2023-10-01T19:39:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable), process ceph-mon, pid 7 2023-10-01T19:39:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: pidfile_write: ignore empty --pid-file 2023-10-01T19:39:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: load: jerasure load: lrc 2023-10-01T19:39:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: RocksDB version: 6.15.5 2023-10-01T19:39:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-10-01T19:39:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Compile date Apr 18 2022 2023-10-01T19:39:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: DB SUMMARY 2023-10-01T19:39:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: DB Session ID: 6G0SFBX4X4ZK99QQ81LA 2023-10-01T19:39:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: CURRENT file: CURRENT 2023-10-01T19:39:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: IDENTITY file: IDENTITY 2023-10-01T19:39:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2023-10-01T19:39:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-10-01T19:39:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-10-01T19:39:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.error_if_exists: 0 2023-10-01T19:39:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.create_if_missing: 0 2023-10-01T19:39:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.paranoid_checks: 1 2023-10-01T19:39:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-01T19:39:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.env: 0x559bf0cec860 2023-10-01T19:39:50.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.fs: Posix File System 2023-10-01T19:39:50.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.info_log: 0x559bf24a9dc0 2023-10-01T19:39:50.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_file_opening_threads: 16 2023-10-01T19:39:50.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.statistics: (nil) 2023-10-01T19:39:50.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.use_fsync: 0 2023-10-01T19:39:50.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_log_file_size: 0 2023-10-01T19:39:50.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-01T19:39:50.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-01T19:39:50.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.keep_log_file_num: 1000 2023-10-01T19:39:50.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.recycle_log_file_num: 0 2023-10-01T19:39:50.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.allow_fallocate: 1 2023-10-01T19:39:50.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.allow_mmap_reads: 0 2023-10-01T19:39:50.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.allow_mmap_writes: 0 2023-10-01T19:39:50.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.use_direct_reads: 0 2023-10-01T19:39:50.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-01T19:39:50.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.create_missing_column_families: 0 2023-10-01T19:39:50.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.db_log_dir: 2023-10-01T19:39:50.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2023-10-01T19:39:50.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-01T19:39:50.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-01T19:39:50.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-01T19:39:50.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-01T19:39:50.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-01T19:39:50.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-01T19:39:50.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.advise_random_on_open: 1 2023-10-01T19:39:50.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.db_write_buffer_size: 0 2023-10-01T19:39:50.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.write_buffer_manager: 0x559bf259a2d0 2023-10-01T19:39:50.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-01T19:39:50.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-10-01T19:39:50.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-01T19:39:50.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-01T19:39:50.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.rate_limiter: (nil) 2023-10-01T19:39:50.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-01T19:39:50.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.wal_recovery_mode: 2 2023-10-01T19:39:50.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.enable_thread_tracking: 0 2023-10-01T19:39:50.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.enable_pipelined_write: 0 2023-10-01T19:39:50.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.unordered_write: 0 2023-10-01T19:39:50.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-01T19:39:50.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-01T19:39:50.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-01T19:39:50.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-01T19:39:50.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.row_cache: None 2023-10-01T19:39:50.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.wal_filter: None 2023-10-01T19:39:50.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-01T19:39:50.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.allow_ingest_behind: 0 2023-10-01T19:39:50.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.preserve_deletes: 0 2023-10-01T19:39:50.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.two_write_queues: 0 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.manual_wal_flush: 0 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.atomic_flush: 0 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.log_readahead_size: 0 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.best_efforts_recovery: 0 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.allow_data_in_errors: 0 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.db_host_id: __hostname__ 2023-10-01T19:39:50.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_background_jobs: 2 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_background_compactions: -1 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_subcompactions: 1 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_total_wal_size: 0 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_open_files: -1 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.bytes_per_sync: 0 2023-10-01T19:39:50.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-01T19:39:50.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-01T19:39:50.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_readahead_size: 0 2023-10-01T19:39:50.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_background_flushes: -1 2023-10-01T19:39:50.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Compression algorithms supported: 2023-10-01T19:39:50.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-01T19:39:50.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: kZSTD supported: 0 2023-10-01T19:39:50.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: kXpressCompression supported: 0 2023-10-01T19:39:50.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: kLZ4HCCompression supported: 1 2023-10-01T19:39:50.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: kLZ4Compression supported: 1 2023-10-01T19:39:50.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: kBZip2Compression supported: 0 2023-10-01T19:39:50.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: kZlibCompression supported: 1 2023-10-01T19:39:50.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: kSnappyCompression supported: 1 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: [db/version_set.cc:4725] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: [db/column_family.cc:597] --------------- Options for column family [default]: 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.merge_operator: 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_filter: None 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 bash[114293]: 965ad47ba3622c817f068b60074cbc16698fc990b880917a96c9c0fa1c9c4ff8 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_filter_factory: None 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.sst_partitioner_factory: None 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559bf2475d00) 2023-10-01T19:39:50.433 INFO:journalctl@ceph.mon.c.smithi057.stdout: cache_index_and_filter_blocks: 1 2023-10-01T19:39:50.434 INFO:journalctl@ceph.mon.c.smithi057.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-01T19:39:50.434 INFO:journalctl@ceph.mon.c.smithi057.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-01T19:39:50.434 INFO:journalctl@ceph.mon.c.smithi057.stdout: pin_top_level_index_and_filter: 1 2023-10-01T19:39:50.434 INFO:journalctl@ceph.mon.c.smithi057.stdout: index_type: 0 2023-10-01T19:39:50.434 INFO:journalctl@ceph.mon.c.smithi057.stdout: data_block_index_type: 0 2023-10-01T19:39:50.434 INFO:journalctl@ceph.mon.c.smithi057.stdout: index_shortening: 1 2023-10-01T19:39:50.434 INFO:journalctl@ceph.mon.c.smithi057.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-01T19:39:50.434 INFO:journalctl@ceph.mon.c.smithi057.stdout: hash_index_allow_collision: 1 2023-10-01T19:39:50.434 INFO:journalctl@ceph.mon.c.smithi057.stdout: checksum: 1 2023-10-01T19:39:50.434 INFO:journalctl@ceph.mon.c.smithi057.stdout: no_block_cache: 0 2023-10-01T19:39:50.434 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_cache: 0x559bf24e0170 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_cache_name: BinnedLRUCache 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_cache_options: 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: capacity : 536870912 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: num_shard_bits : 4 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: strict_capacity_limit : 0 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: high_pri_pool_ratio: 0.000 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_cache_compressed: (nil) 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: persistent_cache: (nil) 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_size: 4096 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_size_deviation: 10 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_restart_interval: 16 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: index_block_restart_interval: 1 2023-10-01T19:39:50.435 INFO:journalctl@ceph.mon.c.smithi057.stdout: metadata_block_size: 4096 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout: partition_filters: 0 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout: use_delta_encoding: 1 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout: whole_key_filtering: 1 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout: verify_compression: 0 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout: read_amp_bytes_per_bit: 0 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout: format_version: 4 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout: enable_index_compression: 1 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_align: 0 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.write_buffer_size: 33554432 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_write_buffer_number: 2 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compression: NoCompression 2023-10-01T19:39:50.436 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.bottommost_compression: Disabled 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.prefix_extractor: nullptr 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.num_levels: 7 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-01T19:39:50.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-01T19:39:50.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-01T19:39:50.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compression_opts.level: 32767 2023-10-01T19:39:50.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compression_opts.strategy: 0 2023-10-01T19:39:50.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-01T19:39:50.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-01T19:39:50.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-01T19:39:50.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compression_opts.enabled: false 2023-10-01T19:39:50.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-01T19:39:50.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-01T19:39:50.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-01T19:39:50.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.target_file_size_base: 67108864 2023-10-01T19:39:50.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-01T19:39:50.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.arena_block_size: 4194304 2023-10-01T19:39:50.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-01T19:39:50.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-01T19:39:50.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-10-01T19:39:50.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.disable_auto_compactions: 0 2023-10-01T19:39:50.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-01T19:39:50.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-01T19:39:50.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-01T19:39:50.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-01T19:39:50.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-01T19:39:50.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-01T19:39:50.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-01T19:39:50.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-01T19:39:50.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-01T19:39:50.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-01T19:39:50.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.table_properties_collectors: 2023-10-01T19:39:50.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.inplace_update_support: 0 2023-10-01T19:39:50.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-01T19:39:50.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-01T19:39:50.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-01T19:39:50.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-01T19:39:50.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.bloom_locality: 0 2023-10-01T19:39:50.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.max_successive_merges: 0 2023-10-01T19:39:50.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-01T19:39:50.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.paranoid_file_checks: 0 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.force_consistency_checks: 1 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.report_bg_io_stats: 0 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.ttl: 2592000 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.enable_blob_files: false 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.min_blob_size: 0 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.blob_file_size: 268435456 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: [db/version_set.cc:4773] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: [db/version_set.cc:4782] Column family [default] (ID 0), log number is 0 2023-10-01T19:39:50.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: [db/version_set.cc:4083] Creating manifest 7 2023-10-01T19:39:50.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696189190148038, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-01T19:39:50.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: [db/db_impl/db_impl_open.cc:847] Recovering log #4 mode 2 2023-10-01T19:39:50.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: [table/block_based/filter_policy.cc:996] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-10-01T19:39:50.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696189190148521, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1696189190, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "286bb91b-5d47-4914-8d12-4e985d44fd57", "db_session_id": "6G0SFBX4X4ZK99QQ81LA"}} 2023-10-01T19:39:50.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: [db/version_set.cc:4083] Creating manifest 9 2023-10-01T19:39:50.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696189190149400, "job": 1, "event": "recovery_finished"} 2023-10-01T19:39:50.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: [file/delete_scheduler.cc:73] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-01T19:39:50.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 systemd[1]: Started Ceph mon.c for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:39:50.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: [db/db_impl/db_impl_open.cc:1701] SstFileManager instance 0x559bf24c6a80 2023-10-01T19:39:50.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: DB pointer 0x559bf24d6000 2023-10-01T19:39:50.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: [db/db_impl/db_impl.cc:902] ------- DUMPING STATS ------- 2023-10-01T19:39:50.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: rocksdb: [db/db_impl/db_impl.cc:903] 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: ** DB Stats ** 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T19:39:50.444 INFO:journalctl@ceph.mon.c.smithi057.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:50.445 INFO:journalctl@ceph.mon.c.smithi057.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:50.445 INFO:journalctl@ceph.mon.c.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:50.445 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T19:39:50.445 INFO:journalctl@ceph.mon.c.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T19:39:50.445 INFO:journalctl@ceph.mon.c.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-01T19:39:50.445 INFO:journalctl@ceph.mon.c.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T19:39:50.445 INFO:journalctl@ceph.mon.c.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:50.445 INFO:journalctl@ceph.mon.c.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T19:39:50.445 INFO:journalctl@ceph.mon.c.smithi057.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-01T19:39:50.445 INFO:journalctl@ceph.mon.c.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-01T19:39:50.445 INFO:journalctl@ceph.mon.c.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-01T19:39:50.445 INFO:journalctl@ceph.mon.c.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-01T19:39:50.446 INFO:journalctl@ceph.mon.c.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-01T19:39:50.446 INFO:journalctl@ceph.mon.c.smithi057.stdout: Cumulative compaction: 0.00 GB write, 0.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T19:39:50.446 INFO:journalctl@ceph.mon.c.smithi057.stdout: Interval compaction: 0.00 GB write, 0.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T19:39:50.446 INFO:journalctl@ceph.mon.c.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-01T19:39:50.446 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T19:39:50.446 INFO:journalctl@ceph.mon.c.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-01T19:39:50.446 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T19:39:50.446 INFO:journalctl@ceph.mon.c.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T19:39:50.446 INFO:journalctl@ceph.mon.c.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-01T19:39:50.446 INFO:journalctl@ceph.mon.c.smithi057.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T19:39:50.446 INFO:journalctl@ceph.mon.c.smithi057.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:50.446 INFO:journalctl@ceph.mon.c.smithi057.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:50.447 INFO:journalctl@ceph.mon.c.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-10-01T19:39:50.447 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T19:39:50.447 INFO:journalctl@ceph.mon.c.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T19:39:50.447 INFO:journalctl@ceph.mon.c.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-01T19:39:50.447 INFO:journalctl@ceph.mon.c.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T19:39:50.447 INFO:journalctl@ceph.mon.c.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:50.447 INFO:journalctl@ceph.mon.c.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T19:39:50.447 INFO:journalctl@ceph.mon.c.smithi057.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-01T19:39:50.447 INFO:journalctl@ceph.mon.c.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-01T19:39:50.447 INFO:journalctl@ceph.mon.c.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-01T19:39:50.447 INFO:journalctl@ceph.mon.c.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-01T19:39:50.447 INFO:journalctl@ceph.mon.c.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout: Cumulative compaction: 0.00 GB write, 0.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: using public_addrv [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: starting mon.c rank -1 at public addrs [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] at bind addrs [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(???) e0 preinit fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).mds e1 new map 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).mds e1 print_map 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout: e1 2023-10-01T19:39:50.448 INFO:journalctl@ceph.mon.c.smithi057.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout: legacy client fscid: -1 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout: No filesystems configured 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-01T19:39:50.449 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-01T19:39:50.450 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-01T19:39:50.450 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mkfs 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:39:50.450 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-01T19:39:50.450 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-01T19:39:50.450 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: monmap e1: 1 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0]} 2023-10-01T19:39:50.450 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: fsmap 2023-10-01T19:39:50.450 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: osdmap e1: 0 total, 0 up, 0 in 2023-10-01T19:39:50.450 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e1: no daemons active 2023-10-01T19:39:50.450 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2570215816' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T19:39:50.450 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3184702918' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-01T19:39:50.450 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3184702918' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-01T19:39:50.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-01T19:39:50.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: monmap e1: 1 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0]} 2023-10-01T19:39:50.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: fsmap 2023-10-01T19:39:50.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: osdmap e1: 0 total, 0 up, 0 in 2023-10-01T19:39:50.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e1: no daemons active 2023-10-01T19:39:50.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3173302710' entity='client.admin' 2023-10-01T19:39:50.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/494171652' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-01T19:39:50.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1390463073' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-01T19:39:50.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2451855033' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-01T19:39:50.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Activating manager daemon y 2023-10-01T19:39:50.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e2: y(active, starting, since 0.00484529s) 2023-10-01T19:39:50.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Manager daemon y is now available 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14100 172.21.15.57:0/2040149989' entity='mgr.y' 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e3: y(active, since 1.00834s) 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/590837300' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-01T19:39:50.452 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e4: y(active, since 3s) 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1908657503' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1908657503' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1136967829' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1136967829' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e5: y(active, since 5s) 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2384895184' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Active manager daemon y restarted 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Activating manager daemon y 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: osdmap e2: 0 total, 0 up, 0 in 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e6: y(active, starting, since 0.453981s) 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:39:50.453 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:39:50.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:39:50.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:39:50.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Manager daemon y is now available 2023-10-01T19:39:50.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:39:50.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:39:50.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e7: y(active, since 1.46582s) 2023-10-01T19:39:50.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: [01/Oct/2023:19:39:06] ENGINE Bus STARTING 2023-10-01T19:39:50.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: [01/Oct/2023:19:39:07] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:39:50.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: [01/Oct/2023:19:39:07] ENGINE Bus STARTED 2023-10-01T19:39:50.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e8: y(active, since 2s) 2023-10-01T19:39:50.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:50.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:50.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:50.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Generating ssh key... 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Deploying cephadm binary to smithi057 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Added host smithi057 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Saving service mon spec with placement count:5 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:50.457 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Saving service mgr spec with placement count:2 2023-10-01T19:39:50.457 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.457 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.457 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3571988148' entity='client.admin' 2023-10-01T19:39:50.457 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.457 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.457 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:39:50.457 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' 2023-10-01T19:39:50.457 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:39:50.457 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14120 172.21.15.57:0/3432854924' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]': finished 2023-10-01T19:39:50.457 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/265524055' entity='client.admin' 2023-10-01T19:39:50.457 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3249544287' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-01T19:39:50.458 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Deploying daemon agent.smithi057 on smithi057 2023-10-01T19:39:50.458 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3249544287' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-01T19:39:50.458 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e9: y(active, since 15s) 2023-10-01T19:39:50.458 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3538706024' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-01T19:39:50.458 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Active manager daemon y restarted 2023-10-01T19:39:50.458 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Activating manager daemon y 2023-10-01T19:39:50.458 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: osdmap e3: 0 total, 0 up, 0 in 2023-10-01T19:39:50.458 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e10: y(active, starting, since 0.0531447s) 2023-10-01T19:39:50.458 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Manager daemon y is now available 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:39:50.459 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:39:50.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-01T19:39:50.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e11: y(active, since 1.05579s) 2023-10-01T19:39:50.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-01T19:39:50.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: [01/Oct/2023:19:39:28] ENGINE Bus STARTING 2023-10-01T19:39:50.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: [01/Oct/2023:19:39:28] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:39:50.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: [01/Oct/2023:19:39:28] ENGINE Bus STARTED 2023-10-01T19:39:50.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Deploying daemon agent.smithi057 on smithi057 2023-10-01T19:39:50.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:50.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.461 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e12: y(active, since 2s) 2023-10-01T19:39:50.461 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:50.461 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.461 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/413849774' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-01T19:39:50.461 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mgrmap e13: y(active, since 6s) 2023-10-01T19:39:50.461 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3208523705' entity='client.admin' 2023-10-01T19:39:50.461 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.461 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.461 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.461 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.461 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.461 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.462 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2830144859' entity='client.admin' 2023-10-01T19:39:50.462 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.462 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.462 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.462 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.462 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.462 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.462 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.462 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:50.462 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.462 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.462 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:50.463 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:39:50.463 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:39:50.463 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:39:50.463 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.463 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.463 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi087", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:50.463 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Deploying cephadm binary to smithi087 2023-10-01T19:39:50.463 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.463 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Added host smithi087 2023-10-01T19:39:50.463 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.463 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:50.463 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]': finished 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Deploying daemon agent.smithi087 on smithi087 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:50.464 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:39:50.465 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:39:50.465 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:39:50.465 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.465 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.465 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.465 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.465 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:50.465 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:39:50.465 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.465 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.465 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1026934141' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-01T19:39:50.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1026934141' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-01T19:39:50.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: osdmap e4: 0 total, 0 up, 0 in 2023-10-01T19:39:50.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:39:50.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:50.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:39:50.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:50.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:39:50.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:50.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi057:[v2:172.21.15.57:3301,v1:172.21.15.57:6790]=c;smithi087:172.21.15.87=b", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:39:50.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Saving service mon spec with placement smithi057:172.21.15.57=a;smithi057:[v2:172.21.15.57:3301,v1:172.21.15.57:6790]=c;smithi087:172.21.15.87=b;count:3 2023-10-01T19:39:50.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: Deploying daemon mon.c on smithi057 2023-10-01T19:39:50.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: from='client.? 172.21.15.87:0/3603447744' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-01T19:39:50.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2023-10-01T19:39:50.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:50 smithi057 ceph-mon[114331]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-10-01T19:39:50.963 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-01T19:39:50.964 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-01T19:39:55.488 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:39:55.488 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":2,"fsid":"0f6d5dc0-6092-11ee-8db5-212e2dc638e7","modified":"2023-10-01T19:39:50.176672Z","created":"2023-10-01T19:38:38.894526Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3301","nonce":0},{"type":"v1","addr":"172.21.15.57:6790","nonce":0}]},"addr":"172.21.15.57:6790/0","public_addr":"172.21.15.57:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-01T19:39:55.490 INFO:teuthology.orchestra.run.smithi087.stderr:dumped monmap epoch 2 2023-10-01T19:39:55.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:39:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: mon.a calling monitor election 2023-10-01T19:39:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:39:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: mon.c calling monitor election 2023-10-01T19:39:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:39:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-01T19:39:55.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: monmap e2: 2 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} 2023-10-01T19:39:55.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: fsmap 2023-10-01T19:39:55.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: osdmap e4: 0 total, 0 up, 0 in 2023-10-01T19:39:55.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: mgrmap e13: y(active, since 27s) 2023-10-01T19:39:55.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: overall HEALTH_OK 2023-10-01T19:39:55.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:55.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:39:55.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:55.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:39:55.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: mon.a calling monitor election 2023-10-01T19:39:55.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:55.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:55.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:39:55.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:55.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: mon.c calling monitor election 2023-10-01T19:39:55.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:55.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:39:55.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:55.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:55.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-01T19:39:55.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: monmap e2: 2 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} 2023-10-01T19:39:55.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: fsmap 2023-10-01T19:39:55.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: osdmap e4: 0 total, 0 up, 0 in 2023-10-01T19:39:55.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: mgrmap e13: y(active, since 27s) 2023-10-01T19:39:55.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: overall HEALTH_OK 2023-10-01T19:39:55.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:55.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:39:55.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:39:56.374 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:56 smithi057 ceph-mon[108286]: Deploying daemon mon.b on smithi087 2023-10-01T19:39:56.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:56 smithi057 ceph-mon[108286]: from='client.? 172.21.15.87:0/775927880' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-01T19:39:56.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:56 smithi057 ceph-mon[108286]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:39:56.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:56 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:56.375 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:56 smithi057 ceph-mon[114331]: Deploying daemon mon.b on smithi087 2023-10-01T19:39:56.375 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:56 smithi057 ceph-mon[114331]: from='client.? 172.21.15.87:0/775927880' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-01T19:39:56.375 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:56 smithi057 ceph-mon[114331]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:39:56.375 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:56 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:39:56.857 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-01T19:39:56.858 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-01T19:39:57.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:39:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:39:56] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:39:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:39:57 smithi057 ceph-mon[108286]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:39:58.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:58.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:58.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:39:57 smithi057 ceph-mon[114331]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:39:58.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 systemd[1]: Started Ceph mon.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:39:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: set uid:gid to 167:167 (ceph:ceph) 2023-10-01T19:39:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable), process ceph-mon, pid 7 2023-10-01T19:39:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: pidfile_write: ignore empty --pid-file 2023-10-01T19:39:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: load: jerasure load: lrc 2023-10-01T19:39:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: RocksDB version: 6.15.5 2023-10-01T19:39:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-10-01T19:39:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Compile date Apr 18 2022 2023-10-01T19:39:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: DB SUMMARY 2023-10-01T19:39:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: DB Session ID: TBHIBDAWZEIBD2R952HL 2023-10-01T19:39:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: CURRENT file: CURRENT 2023-10-01T19:39:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: IDENTITY file: IDENTITY 2023-10-01T19:39:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2023-10-01T19:39:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-01T19:39:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-01T19:39:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.error_if_exists: 0 2023-10-01T19:39:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.create_if_missing: 0 2023-10-01T19:39:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.paranoid_checks: 1 2023-10-01T19:39:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-01T19:39:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.env: 0x557198d3e860 2023-10-01T19:39:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.fs: Posix File System 2023-10-01T19:39:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.info_log: 0x55719ac2bdc0 2023-10-01T19:39:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_file_opening_threads: 16 2023-10-01T19:39:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.statistics: (nil) 2023-10-01T19:39:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.use_fsync: 0 2023-10-01T19:39:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_log_file_size: 0 2023-10-01T19:39:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-01T19:39:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-01T19:39:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.keep_log_file_num: 1000 2023-10-01T19:39:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.recycle_log_file_num: 0 2023-10-01T19:39:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.allow_fallocate: 1 2023-10-01T19:39:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.allow_mmap_reads: 0 2023-10-01T19:39:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.allow_mmap_writes: 0 2023-10-01T19:39:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.use_direct_reads: 0 2023-10-01T19:39:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-01T19:39:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.create_missing_column_families: 0 2023-10-01T19:39:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.db_log_dir: 2023-10-01T19:39:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2023-10-01T19:39:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-01T19:39:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-01T19:39:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-01T19:39:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-01T19:39:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-01T19:39:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-01T19:39:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.advise_random_on_open: 1 2023-10-01T19:39:58.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.db_write_buffer_size: 0 2023-10-01T19:39:58.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.write_buffer_manager: 0x55719ad1c240 2023-10-01T19:39:58.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-01T19:39:58.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-10-01T19:39:58.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-01T19:39:58.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-01T19:39:58.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.rate_limiter: (nil) 2023-10-01T19:39:58.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-01T19:39:58.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.wal_recovery_mode: 2 2023-10-01T19:39:58.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.enable_thread_tracking: 0 2023-10-01T19:39:58.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.enable_pipelined_write: 0 2023-10-01T19:39:58.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.unordered_write: 0 2023-10-01T19:39:58.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-01T19:39:58.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-01T19:39:58.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-01T19:39:58.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-01T19:39:58.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.row_cache: None 2023-10-01T19:39:58.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.wal_filter: None 2023-10-01T19:39:58.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-01T19:39:58.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.allow_ingest_behind: 0 2023-10-01T19:39:58.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.preserve_deletes: 0 2023-10-01T19:39:58.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.two_write_queues: 0 2023-10-01T19:39:58.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.manual_wal_flush: 0 2023-10-01T19:39:58.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.atomic_flush: 0 2023-10-01T19:39:58.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-01T19:39:58.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-01T19:39:58.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-01T19:39:58.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.log_readahead_size: 0 2023-10-01T19:39:58.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-01T19:39:58.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.best_efforts_recovery: 0 2023-10-01T19:39:58.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-01T19:39:58.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-01T19:39:58.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.allow_data_in_errors: 0 2023-10-01T19:39:58.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.db_host_id: __hostname__ 2023-10-01T19:39:58.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_background_jobs: 2 2023-10-01T19:39:58.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_background_compactions: -1 2023-10-01T19:39:58.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_subcompactions: 1 2023-10-01T19:39:58.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-01T19:39:58.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-01T19:39:58.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-01T19:39:58.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_total_wal_size: 0 2023-10-01T19:39:58.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-01T19:39:58.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-01T19:39:58.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-01T19:39:58.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-01T19:39:58.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_open_files: -1 2023-10-01T19:39:58.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.bytes_per_sync: 0 2023-10-01T19:39:58.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-01T19:39:58.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-01T19:39:58.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_readahead_size: 0 2023-10-01T19:39:58.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_background_flushes: -1 2023-10-01T19:39:58.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Compression algorithms supported: 2023-10-01T19:39:58.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-01T19:39:58.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: kZSTD supported: 0 2023-10-01T19:39:58.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: kXpressCompression supported: 0 2023-10-01T19:39:58.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: kLZ4HCCompression supported: 1 2023-10-01T19:39:58.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: kLZ4Compression supported: 1 2023-10-01T19:39:58.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: kBZip2Compression supported: 0 2023-10-01T19:39:58.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: kZlibCompression supported: 1 2023-10-01T19:39:58.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: kSnappyCompression supported: 1 2023-10-01T19:39:58.644 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-01T19:39:58.644 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: [db/version_set.cc:4725] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2023-10-01T19:39:58.644 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: [db/column_family.cc:597] --------------- Options for column family [default]: 2023-10-01T19:39:58.644 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-01T19:39:58.644 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.merge_operator: 2023-10-01T19:39:58.644 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_filter: None 2023-10-01T19:39:58.644 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_filter_factory: None 2023-10-01T19:39:58.644 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.sst_partitioner_factory: None 2023-10-01T19:39:58.644 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-01T19:39:58.644 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-01T19:39:58.644 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55719abf7d00) 2023-10-01T19:39:58.644 INFO:journalctl@ceph.mon.b.smithi087.stdout: cache_index_and_filter_blocks: 1 2023-10-01T19:39:58.645 INFO:journalctl@ceph.mon.b.smithi087.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-01T19:39:58.645 INFO:journalctl@ceph.mon.b.smithi087.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-01T19:39:58.645 INFO:journalctl@ceph.mon.b.smithi087.stdout: pin_top_level_index_and_filter: 1 2023-10-01T19:39:58.645 INFO:journalctl@ceph.mon.b.smithi087.stdout: index_type: 0 2023-10-01T19:39:58.645 INFO:journalctl@ceph.mon.b.smithi087.stdout: data_block_index_type: 0 2023-10-01T19:39:58.645 INFO:journalctl@ceph.mon.b.smithi087.stdout: index_shortening: 1 2023-10-01T19:39:58.645 INFO:journalctl@ceph.mon.b.smithi087.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-01T19:39:58.645 INFO:journalctl@ceph.mon.b.smithi087.stdout: hash_index_allow_collision: 1 2023-10-01T19:39:58.645 INFO:journalctl@ceph.mon.b.smithi087.stdout: checksum: 1 2023-10-01T19:39:58.645 INFO:journalctl@ceph.mon.b.smithi087.stdout: no_block_cache: 0 2023-10-01T19:39:58.645 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_cache: 0x55719ac62170 2023-10-01T19:39:58.645 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_cache_name: BinnedLRUCache 2023-10-01T19:39:58.646 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_cache_options: 2023-10-01T19:39:58.646 INFO:journalctl@ceph.mon.b.smithi087.stdout: capacity : 536870912 2023-10-01T19:39:58.646 INFO:journalctl@ceph.mon.b.smithi087.stdout: num_shard_bits : 4 2023-10-01T19:39:58.646 INFO:journalctl@ceph.mon.b.smithi087.stdout: strict_capacity_limit : 0 2023-10-01T19:39:58.646 INFO:journalctl@ceph.mon.b.smithi087.stdout: high_pri_pool_ratio: 0.000 2023-10-01T19:39:58.646 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_cache_compressed: (nil) 2023-10-01T19:39:58.646 INFO:journalctl@ceph.mon.b.smithi087.stdout: persistent_cache: (nil) 2023-10-01T19:39:58.646 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_size: 4096 2023-10-01T19:39:58.646 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_size_deviation: 10 2023-10-01T19:39:58.646 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_restart_interval: 16 2023-10-01T19:39:58.646 INFO:journalctl@ceph.mon.b.smithi087.stdout: index_block_restart_interval: 1 2023-10-01T19:39:58.646 INFO:journalctl@ceph.mon.b.smithi087.stdout: metadata_block_size: 4096 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout: partition_filters: 0 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout: use_delta_encoding: 1 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout: whole_key_filtering: 1 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout: verify_compression: 0 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout: read_amp_bytes_per_bit: 0 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout: format_version: 4 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout: enable_index_compression: 1 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_align: 0 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.write_buffer_size: 33554432 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_write_buffer_number: 2 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compression: NoCompression 2023-10-01T19:39:58.647 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.bottommost_compression: Disabled 2023-10-01T19:39:58.648 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.prefix_extractor: nullptr 2023-10-01T19:39:58.648 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-01T19:39:58.648 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.num_levels: 7 2023-10-01T19:39:58.648 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-01T19:39:58.648 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-01T19:39:58.648 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-01T19:39:58.648 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-01T19:39:58.648 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-01T19:39:58.648 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-01T19:39:58.648 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-01T19:39:58.648 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-01T19:39:58.648 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-01T19:39:58.649 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-01T19:39:58.649 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-01T19:39:58.649 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compression_opts.level: 32767 2023-10-01T19:39:58.649 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compression_opts.strategy: 0 2023-10-01T19:39:58.649 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-01T19:39:58.649 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-01T19:39:58.649 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-01T19:39:58.649 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compression_opts.enabled: false 2023-10-01T19:39:58.649 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-01T19:39:58.649 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-01T19:39:58.650 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-01T19:39:58.650 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.target_file_size_base: 67108864 2023-10-01T19:39:58.650 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-01T19:39:58.650 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-01T19:39:58.650 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-01T19:39:58.650 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-01T19:39:58.650 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-01T19:39:58.650 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-01T19:39:58.650 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-01T19:39:58.650 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-01T19:39:58.650 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-01T19:39:58.650 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-01T19:39:58.651 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-01T19:39:58.651 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-01T19:39:58.651 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-01T19:39:58.651 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.arena_block_size: 4194304 2023-10-01T19:39:58.651 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-01T19:39:58.651 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-01T19:39:58.651 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-10-01T19:39:58.651 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.disable_auto_compactions: 0 2023-10-01T19:39:58.651 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-01T19:39:58.651 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-01T19:39:58.652 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-01T19:39:58.652 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-01T19:39:58.652 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-01T19:39:58.652 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-01T19:39:58.652 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-01T19:39:58.652 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-01T19:39:58.652 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-01T19:39:58.652 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-01T19:39:58.652 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.table_properties_collectors: 2023-10-01T19:39:58.652 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.inplace_update_support: 0 2023-10-01T19:39:58.652 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-01T19:39:58.653 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-01T19:39:58.653 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-01T19:39:58.653 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-01T19:39:58.653 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.bloom_locality: 0 2023-10-01T19:39:58.653 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.max_successive_merges: 0 2023-10-01T19:39:58.653 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-01T19:39:58.653 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.paranoid_file_checks: 0 2023-10-01T19:39:58.653 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.force_consistency_checks: 1 2023-10-01T19:39:58.653 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.report_bg_io_stats: 0 2023-10-01T19:39:58.653 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.ttl: 2592000 2023-10-01T19:39:58.653 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-01T19:39:58.653 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.enable_blob_files: false 2023-10-01T19:39:58.654 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.min_blob_size: 0 2023-10-01T19:39:58.654 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.blob_file_size: 268435456 2023-10-01T19:39:58.654 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-01T19:39:58.654 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-01T19:39:58.654 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-01T19:39:58.654 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: [db/version_set.cc:4773] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-01T19:39:58.654 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: [db/version_set.cc:4782] Column family [default] (ID 0), log number is 0 2023-10-01T19:39:58.654 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: [db/version_set.cc:4083] Creating manifest 7 2023-10-01T19:39:58.654 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696189198263230, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-01T19:39:58.654 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: [db/db_impl/db_impl_open.cc:847] Recovering log #4 mode 2 2023-10-01T19:39:58.654 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: [table/block_based/filter_policy.cc:996] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-10-01T19:39:58.655 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696189198263612, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1670, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1696189198, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "ac313c2a-1aff-4c07-87af-2970e086f073", "db_session_id": "TBHIBDAWZEIBD2R952HL"}} 2023-10-01T19:39:58.655 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: [db/version_set.cc:4083] Creating manifest 9 2023-10-01T19:39:58.655 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696189198264101, "job": 1, "event": "recovery_finished"} 2023-10-01T19:39:58.655 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: [file/delete_scheduler.cc:73] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-01T19:39:58.655 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: [db/db_impl/db_impl_open.cc:1701] SstFileManager instance 0x55719ac48a80 2023-10-01T19:39:58.655 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: DB pointer 0x55719ac58000 2023-10-01T19:39:58.655 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: [db/db_impl/db_impl.cc:902] ------- DUMPING STATS ------- 2023-10-01T19:39:58.655 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: rocksdb: [db/db_impl/db_impl.cc:903] 2023-10-01T19:39:58.655 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** DB Stats ** 2023-10-01T19:39:58.655 INFO:journalctl@ceph.mon.b.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T19:39:58.655 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-01T19:39:58.655 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** Compaction Stats [default] ** 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: L0 1/0 1.63 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: Sum 1/0 1.63 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T19:39:58.656 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** Compaction Stats [default] ** 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-01T19:39:58.657 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T19:39:58.658 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-01T19:39:58.658 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T19:39:58.658 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** Compaction Stats [default] ** 2023-10-01T19:39:58.658 INFO:journalctl@ceph.mon.b.smithi087.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-01T19:39:58.658 INFO:journalctl@ceph.mon.b.smithi087.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T19:39:58.658 INFO:journalctl@ceph.mon.b.smithi087.stdout: L0 1/0 1.63 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:58.658 INFO:journalctl@ceph.mon.b.smithi087.stdout: Sum 1/0 1.63 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:58.658 INFO:journalctl@ceph.mon.b.smithi087.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-10-01T19:39:58.658 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T19:39:58.658 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** Compaction Stats [default] ** 2023-10-01T19:39:58.658 INFO:journalctl@ceph.mon.b.smithi087.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-01T19:39:58.658 INFO:journalctl@ceph.mon.b.smithi087.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T19:39:58.659 INFO:journalctl@ceph.mon.b.smithi087.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 2023-10-01T19:39:58.659 INFO:journalctl@ceph.mon.b.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T19:39:58.659 INFO:journalctl@ceph.mon.b.smithi087.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-01T19:39:58.659 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-01T19:39:58.659 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-01T19:39:58.659 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-01T19:39:58.659 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-01T19:39:58.659 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T19:39:58.659 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T19:39:58.659 INFO:journalctl@ceph.mon.b.smithi087.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-01T19:39:58.659 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T19:39:58.659 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-01T19:39:58.660 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-01T19:39:58.660 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: using public_addr v2:172.21.15.87:0/0 -> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] 2023-10-01T19:39:58.660 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: starting mon.b rank -1 at public addrs [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] at bind addrs [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:39:58.660 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(???) e0 preinit fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:39:58.660 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).mds e1 new map 2023-10-01T19:39:58.660 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-01T19:39:58.660 INFO:journalctl@ceph.mon.b.smithi087.stdout: e1 2023-10-01T19:39:58.660 INFO:journalctl@ceph.mon.b.smithi087.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-01T19:39:58.660 INFO:journalctl@ceph.mon.b.smithi087.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-01T19:39:58.660 INFO:journalctl@ceph.mon.b.smithi087.stdout: legacy client fscid: -1 2023-10-01T19:39:58.660 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T19:39:58.660 INFO:journalctl@ceph.mon.b.smithi087.stdout: No filesystems configured 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:39:58.661 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:39:58.662 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2023-10-01T19:39:58.662 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:39:58 smithi087 ceph-mon[109386]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-10-01T19:40:03.342 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-10-01T19:40:03.342 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":3,"fsid":"0f6d5dc0-6092-11ee-8db5-212e2dc638e7","modified":"2023-10-01T19:39:58.294342Z","created":"2023-10-01T19:38:38.894526Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3301","nonce":0},{"type":"v1","addr":"172.21.15.57:6790","nonce":0}]},"addr":"172.21.15.57:6790/0","public_addr":"172.21.15.57:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:3300","nonce":0},{"type":"v1","addr":"172.21.15.87:6789","nonce":0}]},"addr":"172.21.15.87:6789/0","public_addr":"172.21.15.87:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-10-01T19:40:03.345 INFO:teuthology.orchestra.run.smithi087.stderr:dumped monmap epoch 3 2023-10-01T19:40:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:40:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: mon.a calling monitor election 2023-10-01T19:40:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: mon.c calling monitor election 2023-10-01T19:40:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:40:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='client.? 172.21.15.87:0/3280086397' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-01T19:40:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: mon.b calling monitor election 2023-10-01T19:40:03.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:03.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T19:40:03.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} 2023-10-01T19:40:03.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: fsmap 2023-10-01T19:40:03.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: osdmap e4: 0 total, 0 up, 0 in 2023-10-01T19:40:03.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: mgrmap e13: y(active, since 35s) 2023-10-01T19:40:03.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: overall HEALTH_OK 2023-10-01T19:40:03.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:03.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:03.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:03.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:03 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:03.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:40:03.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: mon.a calling monitor election 2023-10-01T19:40:03.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: mon.c calling monitor election 2023-10-01T19:40:03.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:40:03.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='client.? 172.21.15.87:0/3280086397' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-01T19:40:03.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:03.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: mon.b calling monitor election 2023-10-01T19:40:03.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:03.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T19:40:03.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} 2023-10-01T19:40:03.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: fsmap 2023-10-01T19:40:03.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: osdmap e4: 0 total, 0 up, 0 in 2023-10-01T19:40:03.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: mgrmap e13: y(active, since 35s) 2023-10-01T19:40:03.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: overall HEALTH_OK 2023-10-01T19:40:03.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:03.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:03.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:03.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:03.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:40:03.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: mon.a calling monitor election 2023-10-01T19:40:03.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: mon.c calling monitor election 2023-10-01T19:40:03.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:40:03.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='client.? 172.21.15.87:0/3280086397' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-01T19:40:03.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:03.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: mon.b calling monitor election 2023-10-01T19:40:03.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:03.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:03.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T19:40:03.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} 2023-10-01T19:40:03.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: fsmap 2023-10-01T19:40:03.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: osdmap e4: 0 total, 0 up, 0 in 2023-10-01T19:40:03.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: mgrmap e13: y(active, since 35s) 2023-10-01T19:40:03.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: overall HEALTH_OK 2023-10-01T19:40:03.666 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:03.666 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:03.666 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:03.666 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:03 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:03.698 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-01T19:40:03.698 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-01T19:40:04.776 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:40:04.776 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:40:04.777 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:04.777 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:04.777 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:04.777 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:04.777 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:40:04.777 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: Reconfiguring mon.a (unknown last config time)... 2023-10-01T19:40:04.777 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:40:04.777 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T19:40:04.777 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:04.777 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: Reconfiguring daemon mon.a on smithi057 2023-10-01T19:40:04.777 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:04.778 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:40:04.778 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:40:04.778 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:04.778 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:04.778 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:04.778 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:04.778 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:40:04.778 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: Reconfiguring mon.a (unknown last config time)... 2023-10-01T19:40:04.778 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:40:04.778 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T19:40:04.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:04.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: Reconfiguring daemon mon.a on smithi057 2023-10-01T19:40:04.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:04.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:40:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:40:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:40:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: Reconfiguring mon.a (unknown last config time)... 2023-10-01T19:40:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:40:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T19:40:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:04.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: Reconfiguring daemon mon.a on smithi057 2023-10-01T19:40:04.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:40:05.645 INFO:teuthology.orchestra.run.smithi057.stdout:# minimal ceph.conf for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:40:05.645 INFO:teuthology.orchestra.run.smithi057.stdout:[global] 2023-10-01T19:40:05.645 INFO:teuthology.orchestra.run.smithi057.stdout: fsid = 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:40:05.645 INFO:teuthology.orchestra.run.smithi057.stdout: mon_host = [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] 2023-10-01T19:40:05.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:40:05 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:40:05] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:40:05.978 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:40:05 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:40:05] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:40:06.244 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:40:05 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:40:05] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:40:06.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: Reconfiguring mon.c (monmap changed)... 2023-10-01T19:40:06.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:40:06.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T19:40:06.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:06.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: Reconfiguring daemon mon.c on smithi057 2023-10-01T19:40:06.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:06.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/924776063' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:06.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: Reconfiguring mon.c (monmap changed)... 2023-10-01T19:40:06.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:40:06.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T19:40:06.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:06.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: Reconfiguring daemon mon.c on smithi057 2023-10-01T19:40:06.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:06.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/924776063' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:06.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:06 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.263 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-01T19:40:06.264 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-01T19:40:06.264 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.conf 2023-10-01T19:40:06.300 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-01T19:40:06.300 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:40:06.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: Reconfiguring mon.c (monmap changed)... 2023-10-01T19:40:06.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:40:06.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T19:40:06.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:06.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: Reconfiguring daemon mon.c on smithi057 2023-10-01T19:40:06.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:06.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/924776063' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:06.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:06 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:06.374 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-10-01T19:40:06.374 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.conf 2023-10-01T19:40:06.408 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-10-01T19:40:06.408 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:40:06.477 INFO:tasks.cephadm:Adding mgr.y on smithi057 2023-10-01T19:40:06.477 INFO:tasks.cephadm:Adding mgr.x on smithi087 2023-10-01T19:40:06.478 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch apply mgr '2;smithi057=y;smithi087=x' 2023-10-01T19:40:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:07 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:07 smithi087 ceph-mon[109386]: Reconfiguring mon.b (monmap changed)... 2023-10-01T19:40:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:07 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:40:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:07 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T19:40:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:07 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:07 smithi087 ceph-mon[109386]: Reconfiguring daemon mon.b on smithi087 2023-10-01T19:40:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:07 smithi087 ceph-mon[109386]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:07 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:08.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:08.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[108286]: Reconfiguring mon.b (monmap changed)... 2023-10-01T19:40:08.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:40:08.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T19:40:08.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:08.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[108286]: Reconfiguring daemon mon.b on smithi087 2023-10-01T19:40:08.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[108286]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:08.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:08.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:08.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[114331]: Reconfiguring mon.b (monmap changed)... 2023-10-01T19:40:08.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T19:40:08.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T19:40:08.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:08.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[114331]: Reconfiguring daemon mon.b on smithi087 2023-10-01T19:40:08.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[114331]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:08.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:07 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:08.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:40:08 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:40:08] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:40:08.734 INFO:teuthology.orchestra.run.smithi087.stdout:Scheduled mgr update... 2023-10-01T19:40:09.109 DEBUG:teuthology.orchestra.run.smithi087:mgr.x> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.x.service 2023-10-01T19:40:09.111 INFO:tasks.cephadm:Deploying OSDs... 2023-10-01T19:40:09.111 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-01T19:40:09.111 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/scratch_devs of=/dev/stdout 2023-10-01T19:40:09.137 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-01T19:40:09.137 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_1 2023-10-01T19:40:09.140 INFO:journalctl@ceph.mgr.x.smithi087.stdout:-- Logs begin at Sun 2023-10-01 19:27:12 UTC. -- 2023-10-01T19:40:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-01T19:40:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-01T19:40:09.201 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 384780 Links: 1 2023-10-01T19:40:09.201 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-01T19:40:09.201 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-10-01T19:40:09.201 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-01 19:40:07.695218842 +0000 2023-10-01T19:40:09.201 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-01 19:36:46.739422341 +0000 2023-10-01T19:40:09.202 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-01 19:36:46.739422341 +0000 2023-10-01T19:40:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-10-01T19:40:09.202 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-01T19:40:09.281 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-01T19:40:09.281 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-01T19:40:09.281 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000323132 s, 1.6 MB/s 2023-10-01T19:40:09.282 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-01T19:40:09.348 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_2 2023-10-01T19:40:09.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:40:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-01T19:40:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T19:40:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-01T19:40:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-01T19:40:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 382755 Links: 1 2023-10-01T19:40:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-01T19:40:09.412 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-10-01T19:40:09.412 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-01 19:40:07.695218842 +0000 2023-10-01T19:40:09.412 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-01 19:36:47.344413524 +0000 2023-10-01T19:40:09.412 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-01 19:36:47.344413524 +0000 2023-10-01T19:40:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-10-01T19:40:09.412 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-01T19:40:09.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:09.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:09.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:09.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:09.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:09.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:09.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:40:09.485 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-01T19:40:09.485 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T19:40:09.485 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:09.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:09.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:09.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:09.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:09.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:09.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:09.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:09.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:40:09.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-01T19:40:09.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T19:40:09.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-01T19:40:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-01T19:40:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000309928 s, 1.7 MB/s 2023-10-01T19:40:09.488 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-01T19:40:09.553 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_3 2023-10-01T19:40:09.616 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-01T19:40:09.616 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-01T19:40:09.616 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 382863 Links: 1 2023-10-01T19:40:09.616 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-01T19:40:09.616 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-10-01T19:40:09.616 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-01 19:40:07.695218842 +0000 2023-10-01T19:40:09.616 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-01 19:36:47.941404823 +0000 2023-10-01T19:40:09.617 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-01 19:36:47.941404823 +0000 2023-10-01T19:40:09.617 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-10-01T19:40:09.617 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-01T19:40:09.692 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-01T19:40:09.692 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-01T19:40:09.692 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000387122 s, 1.3 MB/s 2023-10-01T19:40:09.693 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-01T19:40:09.758 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_4 2023-10-01T19:40:09.822 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-01T19:40:09.823 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-01T19:40:09.823 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 393125 Links: 1 2023-10-01T19:40:09.823 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-01T19:40:09.823 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-10-01T19:40:09.823 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-01 19:40:07.695218842 +0000 2023-10-01T19:40:09.823 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-01 19:36:48.524396327 +0000 2023-10-01T19:40:09.823 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-01 19:36:48.524396327 +0000 2023-10-01T19:40:09.823 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-10-01T19:40:09.823 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-01T19:40:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-01T19:40:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-01T19:40:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000306202 s, 1.7 MB/s 2023-10-01T19:40:09.898 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-01T19:40:09.963 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-10-01T19:40:09.963 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/scratch_devs of=/dev/stdout 2023-10-01T19:40:09.982 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-01T19:40:09.982 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_1 2023-10-01T19:40:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-01T19:40:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-01T19:40:10.041 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 390634 Links: 1 2023-10-01T19:40:10.041 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-01T19:40:10.041 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2023-10-01T19:40:10.041 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2023-10-01 19:40:09.335083692 +0000 2023-10-01T19:40:10.042 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2023-10-01 19:36:05.559726069 +0000 2023-10-01T19:40:10.042 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2023-10-01 19:36:05.559726069 +0000 2023-10-01T19:40:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2023-10-01T19:40:10.042 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-01T19:40:10.111 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2023-10-01T19:40:10.111 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2023-10-01T19:40:10.111 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000249722 s, 2.1 MB/s 2023-10-01T19:40:10.112 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-01T19:40:10.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:40:09 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:40:09] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:40:10.173 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_2 2023-10-01T19:40:10.231 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-01T19:40:10.231 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-01T19:40:10.231 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 395387 Links: 1 2023-10-01T19:40:10.231 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-01T19:40:10.231 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2023-10-01T19:40:10.231 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2023-10-01 19:40:09.335083692 +0000 2023-10-01T19:40:10.231 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2023-10-01 19:36:06.068714702 +0000 2023-10-01T19:40:10.231 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2023-10-01 19:36:06.068714702 +0000 2023-10-01T19:40:10.232 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2023-10-01T19:40:10.232 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-01T19:40:10.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2023-10-01T19:40:10.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2023-10-01T19:40:10.298 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000220346 s, 2.3 MB/s 2023-10-01T19:40:10.298 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-01T19:40:10.358 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_3 2023-10-01T19:40:10.417 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-01T19:40:10.417 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-01T19:40:10.418 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 396740 Links: 1 2023-10-01T19:40:10.418 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-01T19:40:10.418 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2023-10-01T19:40:10.418 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2023-10-01 19:40:10.416059600 +0000 2023-10-01T19:40:10.418 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2023-10-01 19:36:06.576703356 +0000 2023-10-01T19:40:10.418 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2023-10-01 19:36:06.576703356 +0000 2023-10-01T19:40:10.418 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2023-10-01T19:40:10.418 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-01T19:40:10.492 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2023-10-01T19:40:10.492 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2023-10-01T19:40:10.492 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000199969 s, 2.6 MB/s 2023-10-01T19:40:10.493 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-01T19:40:10.552 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_4 2023-10-01T19:40:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-01T19:40:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-01T19:40:10.612 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 395484 Links: 1 2023-10-01T19:40:10.612 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-01T19:40:10.612 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2023-10-01T19:40:10.612 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2023-10-01 19:40:10.553056546 +0000 2023-10-01T19:40:10.612 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2023-10-01 19:36:07.110691430 +0000 2023-10-01T19:40:10.613 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2023-10-01 19:36:07.110691430 +0000 2023-10-01T19:40:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2023-10-01T19:40:10.613 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-01T19:40:10.632 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:10 smithi087 systemd[1]: Starting Ceph mgr.x for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:40:10.663 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2023-10-01T19:40:10.663 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2023-10-01T19:40:10.664 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000219902 s, 2.3 MB/s 2023-10-01T19:40:10.664 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-01T19:40:10.725 INFO:tasks.cephadm:Deploying osd.0 on smithi057 with /dev/vg_nvme/lv_4... 2023-10-01T19:40:10.725 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-01T19:40:10.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:10 smithi057 ceph-mon[114331]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi057=y;smithi087=x", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:40:10.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:10 smithi057 ceph-mon[114331]: Saving service mgr spec with placement smithi057=y;smithi087=x;count:2 2023-10-01T19:40:10.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:10 smithi057 ceph-mon[114331]: Deploying daemon mgr.x on smithi087 2023-10-01T19:40:10.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:10 smithi057 ceph-mon[114331]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:10.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:10.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:10 smithi057 ceph-mon[108286]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi057=y;smithi087=x", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:40:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:10 smithi057 ceph-mon[108286]: Saving service mgr spec with placement smithi057=y;smithi087=x;count:2 2023-10-01T19:40:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:10 smithi057 ceph-mon[108286]: Deploying daemon mgr.x on smithi087 2023-10-01T19:40:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:10 smithi057 ceph-mon[108286]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:10.948 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:10 smithi087 ceph-mon[109386]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi057=y;smithi087=x", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:40:10.948 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:10 smithi087 ceph-mon[109386]: Saving service mgr spec with placement smithi057=y;smithi087=x;count:2 2023-10-01T19:40:10.948 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:10 smithi087 ceph-mon[109386]: Deploying daemon mgr.x on smithi087 2023-10-01T19:40:10.948 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:10 smithi087 ceph-mon[109386]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:10.948 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:10.949 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:11.237 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:10 smithi087 bash[111129]: d46b745be5baf5a62ab8451b63ac0e04631d4a670dbb9e361105278926fc8fa1 2023-10-01T19:40:11.237 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:10 smithi087 ceph-mgr[111192]: -- 172.21.15.87:0/186138637 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fe9e758340 con 0x55fe9e730c00 2023-10-01T19:40:11.237 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:10 smithi087 systemd[1]: Started Ceph mgr.x for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:40:11.237 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:11 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:11.075+0000 7fee7b3b6000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T19:40:11.237 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:11 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:11.236+0000 7fee7b3b6000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T19:40:11.769 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:11 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:11.568+0000 7fee7b3b6000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T19:40:12.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:40:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-mon[109386]: Reconfiguring mgr.y (unknown last config time)... 2023-10-01T19:40:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:40:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T19:40:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-mon[109386]: Reconfiguring daemon mgr.y on smithi057 2023-10-01T19:40:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-mon[109386]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:12.317 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:12.317 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:12.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:12.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:12.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:12.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:40:12.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[108286]: Reconfiguring mgr.y (unknown last config time)... 2023-10-01T19:40:12.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:40:12.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T19:40:12.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:12.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[108286]: Reconfiguring daemon mgr.y on smithi057 2023-10-01T19:40:12.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[108286]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:12.319 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:12.319 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:12.319 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:12.319 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:12.319 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:12.319 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:40:12.319 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[114331]: Reconfiguring mgr.y (unknown last config time)... 2023-10-01T19:40:12.319 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:40:12.319 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T19:40:12.319 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:12.320 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[114331]: Reconfiguring daemon mgr.y on smithi057 2023-10-01T19:40:12.320 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-mon[114331]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:12.421 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-01T19:40:12.421 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-01T19:40:12.421 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2023-10-01T19:40:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2023-10-01T19:40:12.422 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118144 s, 888 MB/s 2023-10-01T19:40:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2023-10-01T19:40:12.545 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:12.138+0000 7fee7b3b6000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T19:40:12.857 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi057:vg_nvme/lv_4 2023-10-01T19:40:12.882 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:12 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:12.544+0000 7fee7b3b6000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T19:40:13.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:40:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:40:12] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:40:13.481 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.481 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:13.481 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:13.481 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:13.481 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.481 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:40:13.481 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.481 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.482 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.482 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:13.482 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:13.482 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:13.482 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.482 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.482 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:13.482 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:13.482 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:13.482 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.482 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:40:13.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:13.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:13.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:13.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.632 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:13.256+0000 7fee7b3b6000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T19:40:13.633 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:13.428+0000 7fee7b3b6000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T19:40:13.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:13.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:13.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:13.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:40:13.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:13.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:13.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:13.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:13.859 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:40:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:40:13] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:40:14.286 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:14.026+0000 7fee7b3b6000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T19:40:14.286 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:14.108+0000 7fee7b3b6000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T19:40:14.287 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:14.194+0000 7fee7b3b6000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T19:40:14.632 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:14.285+0000 7fee7b3b6000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T19:40:14.633 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:14.374+0000 7fee7b3b6000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T19:40:14.633 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:14.466+0000 7fee7b3b6000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T19:40:14.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:14.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:14.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-mon[109386]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:14.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:40:14.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:40:14.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:14.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:14.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:14.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:14 smithi057 ceph-mon[108286]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:14.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:40:14.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:40:14.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:14.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:14.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:14.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:14 smithi057 ceph-mon[114331]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:14.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:40:14.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:40:14.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:15.037 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:14.765+0000 7fee7b3b6000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T19:40:15.037 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:14.858+0000 7fee7b3b6000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T19:40:15.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:15 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:15.036+0000 7fee7b3b6000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T19:40:15.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:15 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:15.119+0000 7fee7b3b6000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T19:40:15.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:15 smithi057 ceph-mon[108286]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:40:15.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:15 smithi057 ceph-mon[114331]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:40:15.959 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:15 smithi087 ceph-mon[109386]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:40:15.959 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:15 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:15.810+0000 7fee7b3b6000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T19:40:16.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:15 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:15.957+0000 7fee7b3b6000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T19:40:16.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:16.090+0000 7fee7b3b6000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:40:16.882 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:16.476+0000 7fee7b3b6000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T19:40:16.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:16.610+0000 7fee7b3b6000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:40:16.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:16 smithi087 ceph-mon[109386]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:16.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:16 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/158763170' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c7f9d93-03f1-48e1-9ac3-eab5bbd20afa"}]: dispatch 2023-10-01T19:40:16.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:16 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c7f9d93-03f1-48e1-9ac3-eab5bbd20afa"}]: dispatch 2023-10-01T19:40:16.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:16 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c7f9d93-03f1-48e1-9ac3-eab5bbd20afa"}]': finished 2023-10-01T19:40:16.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:16 smithi087 ceph-mon[109386]: osdmap e5: 1 total, 0 up, 1 in 2023-10-01T19:40:16.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:16 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:17.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:16 smithi057 ceph-mon[108286]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:17.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:16 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/158763170' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c7f9d93-03f1-48e1-9ac3-eab5bbd20afa"}]: dispatch 2023-10-01T19:40:17.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:16 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c7f9d93-03f1-48e1-9ac3-eab5bbd20afa"}]: dispatch 2023-10-01T19:40:17.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:16 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c7f9d93-03f1-48e1-9ac3-eab5bbd20afa"}]': finished 2023-10-01T19:40:17.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:16 smithi057 ceph-mon[108286]: osdmap e5: 1 total, 0 up, 1 in 2023-10-01T19:40:17.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:16 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:17.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:16 smithi057 ceph-mon[114331]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:17.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:16 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/158763170' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c7f9d93-03f1-48e1-9ac3-eab5bbd20afa"}]: dispatch 2023-10-01T19:40:17.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:16 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c7f9d93-03f1-48e1-9ac3-eab5bbd20afa"}]: dispatch 2023-10-01T19:40:17.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:16 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c7f9d93-03f1-48e1-9ac3-eab5bbd20afa"}]': finished 2023-10-01T19:40:17.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:16 smithi057 ceph-mon[114331]: osdmap e5: 1 total, 0 up, 1 in 2023-10-01T19:40:17.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:16 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:17.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:16.989+0000 7fee7b3b6000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T19:40:17.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:40:17 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:40:17.102+0000 7fee7b3b6000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T19:40:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:17 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1670311279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:40:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:17 smithi087 ceph-mon[109386]: Standby manager daemon x started 2023-10-01T19:40:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:17 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/2110721811' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T19:40:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:17 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/2110721811' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:40:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:17 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/2110721811' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T19:40:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:17 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/2110721811' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:40:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:17 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1670311279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:40:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:17 smithi057 ceph-mon[108286]: Standby manager daemon x started 2023-10-01T19:40:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:17 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/2110721811' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T19:40:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:17 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/2110721811' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:40:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:17 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/2110721811' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T19:40:18.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:17 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/2110721811' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:40:18.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:17 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1670311279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:40:18.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:17 smithi057 ceph-mon[114331]: Standby manager daemon x started 2023-10-01T19:40:18.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:17 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/2110721811' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T19:40:18.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:17 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/2110721811' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:40:18.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:17 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/2110721811' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T19:40:18.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:17 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/2110721811' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:40:19.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:18 smithi057 ceph-mon[108286]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:19.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:18 smithi057 ceph-mon[108286]: mgrmap e14: y(active, since 50s), standbys: x 2023-10-01T19:40:19.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:18 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T19:40:19.098 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:18 smithi057 ceph-mon[114331]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:19.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:18 smithi057 ceph-mon[114331]: mgrmap e14: y(active, since 50s), standbys: x 2023-10-01T19:40:19.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:18 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T19:40:19.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:18 smithi087 ceph-mon[109386]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:19.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:18 smithi087 ceph-mon[109386]: mgrmap e14: y(active, since 50s), standbys: x 2023-10-01T19:40:19.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:18 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T19:40:20.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:20 smithi057 ceph-mon[108286]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:20.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:20 smithi057 ceph-mon[114331]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:21.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:20 smithi087 ceph-mon[109386]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:22.061 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:21 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-01T19:40:22.061 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:21 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:22.062 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:21 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-01T19:40:22.062 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:21 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:22.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:21 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-01T19:40:22.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:21 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:22.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:22 smithi057 ceph-mon[108286]: Deploying daemon osd.0 on smithi057 2023-10-01T19:40:22.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:22 smithi057 ceph-mon[108286]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:22.993 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:22 smithi057 ceph-mon[114331]: Deploying daemon osd.0 on smithi057 2023-10-01T19:40:22.993 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:22 smithi057 ceph-mon[114331]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:23.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:22 smithi087 ceph-mon[109386]: Deploying daemon osd.0 on smithi057 2023-10-01T19:40:23.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:22 smithi087 ceph-mon[109386]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:23.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:23 smithi057 ceph-mon[108286]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:23.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:23 smithi057 ceph-mon[114331]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:24.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:23 smithi087 ceph-mon[109386]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:26.361 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:26.361 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:26.361 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:26.361 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:26.361 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:26.361 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-mon[108286]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:26.362 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:26.362 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:26.362 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:26.362 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:26.362 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:26.362 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-mon[114331]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:26.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:26 smithi087 ceph-mon[109386]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:27.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:40:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:40:26] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:40:28.017 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 0 on host 'smithi057' 2023-10-01T19:40:28.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:27 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:28.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:27 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:28.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:27 smithi057 ceph-mon[108286]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:28.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:27 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:40:28.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:27 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:40:28.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:27 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:28.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:27 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:28.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:27 smithi057 ceph-mon[114331]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:28.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:27 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:40:28.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:27 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:40:28.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:27 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:28.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:27 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:28.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:27 smithi087 ceph-mon[109386]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:28.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:27 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:40:28.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:27 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:40:28.561 DEBUG:teuthology.orchestra.run.smithi057:osd.0> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.0.service 2023-10-01T19:40:28.564 INFO:tasks.cephadm:Deploying osd.1 on smithi057 with /dev/vg_nvme/lv_3... 2023-10-01T19:40:28.564 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-01T19:40:28.599 INFO:journalctl@ceph.osd.0.smithi057.stdout:-- Logs begin at Sun 2023-10-01 19:27:22 UTC. -- 2023-10-01T19:40:29.205 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:29.205 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:29.205 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:29.205 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:29.206 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:40:29.206 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[108286]: Adjusting osd_memory_target on smithi057 to 15070M 2023-10-01T19:40:29.206 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:29.206 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:29.206 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[108286]: from='osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T19:40:29.206 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:29.206 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:29.206 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:29.206 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:29.207 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:40:29.207 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[114331]: Adjusting osd_memory_target on smithi057 to 15070M 2023-10-01T19:40:29.207 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:29.207 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:29.207 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:28 smithi057 ceph-mon[114331]: from='osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T19:40:29.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:28 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:28 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:28 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:28 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:28 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:40:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:28 smithi087 ceph-mon[109386]: Adjusting osd_memory_target on smithi057 to 15070M 2023-10-01T19:40:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:28 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:28 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:28 smithi087 ceph-mon[109386]: from='osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T19:40:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:30 smithi087 ceph-mon[109386]: from='osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-01T19:40:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:30 smithi087 ceph-mon[109386]: osdmap e6: 1 total, 0 up, 1 in 2023-10-01T19:40:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:30 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:30 smithi087 ceph-mon[109386]: from='osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:40:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:30 smithi087 ceph-mon[109386]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:30 smithi057 ceph-mon[108286]: from='osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-01T19:40:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:30 smithi057 ceph-mon[108286]: osdmap e6: 1 total, 0 up, 1 in 2023-10-01T19:40:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:30 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:30 smithi057 ceph-mon[108286]: from='osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:40:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:30 smithi057 ceph-mon[108286]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:30.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:30 smithi057 ceph-mon[114331]: from='osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-01T19:40:30.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:30 smithi057 ceph-mon[114331]: osdmap e6: 1 total, 0 up, 1 in 2023-10-01T19:40:30.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:30 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:30.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:30 smithi057 ceph-mon[114331]: from='osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:40:30.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:30 smithi057 ceph-mon[114331]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-01T19:40:30.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 19:40:30 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[118280]: 2023-10-01T19:40:30.077+0000 7f8f84890700 -1 osd.0 0 waiting for initial osdmap 2023-10-01T19:40:30.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 19:40:30 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[118280]: 2023-10-01T19:40:30.080+0000 7f8f7f227700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-01T19:40:30.551 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-01T19:40:30.551 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-01T19:40:30.551 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2023-10-01T19:40:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2023-10-01T19:40:30.552 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0159476 s, 658 MB/s 2023-10-01T19:40:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2023-10-01T19:40:30.981 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi057:vg_nvme/lv_3 2023-10-01T19:40:31.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:31 smithi057 ceph-mon[114331]: from='osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-01T19:40:31.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:31 smithi057 ceph-mon[114331]: osdmap e7: 1 total, 0 up, 1 in 2023-10-01T19:40:31.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:31 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:31.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:31 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:31.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:31 smithi057 ceph-mon[108286]: from='osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-01T19:40:31.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:31 smithi057 ceph-mon[108286]: osdmap e7: 1 total, 0 up, 1 in 2023-10-01T19:40:31.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:31 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:31.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:31 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:31.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:31 smithi087 ceph-mon[109386]: from='osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-01T19:40:31.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:31 smithi087 ceph-mon[109386]: osdmap e7: 1 total, 0 up, 1 in 2023-10-01T19:40:31.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:31 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:31.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:31 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:32.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:32 smithi087 ceph-mon[109386]: purged_snaps scrub starts 2023-10-01T19:40:32.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:32 smithi087 ceph-mon[109386]: purged_snaps scrub ok 2023-10-01T19:40:32.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:32 smithi087 ceph-mon[109386]: osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208] boot 2023-10-01T19:40:32.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:32 smithi087 ceph-mon[109386]: osdmap e8: 1 total, 1 up, 1 in 2023-10-01T19:40:32.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:32 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:32.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:32 smithi087 ceph-mon[109386]: pgmap v30: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:32.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-mon[108286]: purged_snaps scrub starts 2023-10-01T19:40:32.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-mon[108286]: purged_snaps scrub ok 2023-10-01T19:40:32.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-mon[108286]: osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208] boot 2023-10-01T19:40:32.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-mon[108286]: osdmap e8: 1 total, 1 up, 1 in 2023-10-01T19:40:32.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:32.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-mon[108286]: pgmap v30: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:32.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-mon[114331]: purged_snaps scrub starts 2023-10-01T19:40:32.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-mon[114331]: purged_snaps scrub ok 2023-10-01T19:40:32.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-mon[114331]: osd.0 [v2:172.21.15.57:6802/3459964208,v1:172.21.15.57:6803/3459964208] boot 2023-10-01T19:40:32.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-mon[114331]: osdmap e8: 1 total, 1 up, 1 in 2023-10-01T19:40:32.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:40:32.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-mon[114331]: pgmap v30: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:33.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:40:32 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:40:32] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:40:33.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[108286]: osdmap e9: 1 total, 1 up, 1 in 2023-10-01T19:40:33.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[108286]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:40:33.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:40:33.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:40:33.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:33.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:33.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:33.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[114331]: osdmap e9: 1 total, 1 up, 1 in 2023-10-01T19:40:33.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[114331]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:40:33.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:40:33.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:40:33.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:33.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:33.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:33 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:33 smithi087 ceph-mon[109386]: osdmap e9: 1 total, 1 up, 1 in 2023-10-01T19:40:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:33 smithi087 ceph-mon[109386]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:40:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:33 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:40:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:33 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:40:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:33 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:33 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:33 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:34.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:34 smithi087 ceph-mon[109386]: pgmap v32: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:34 smithi057 ceph-mon[108286]: pgmap v32: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:34.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:34 smithi057 ceph-mon[114331]: pgmap v32: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:35 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1977659733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2272344b-6026-4653-ab79-e9c746993364"}]: dispatch 2023-10-01T19:40:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:35 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2272344b-6026-4653-ab79-e9c746993364"}]: dispatch 2023-10-01T19:40:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:35 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2272344b-6026-4653-ab79-e9c746993364"}]': finished 2023-10-01T19:40:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:35 smithi087 ceph-mon[109386]: osdmap e10: 2 total, 1 up, 2 in 2023-10-01T19:40:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:35 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:35 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3143630396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:40:35.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:35 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1977659733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2272344b-6026-4653-ab79-e9c746993364"}]: dispatch 2023-10-01T19:40:35.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:35 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2272344b-6026-4653-ab79-e9c746993364"}]: dispatch 2023-10-01T19:40:35.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:35 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2272344b-6026-4653-ab79-e9c746993364"}]': finished 2023-10-01T19:40:35.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:35 smithi057 ceph-mon[108286]: osdmap e10: 2 total, 1 up, 2 in 2023-10-01T19:40:35.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:35 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:35.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:35 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3143630396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:40:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:35 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1977659733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2272344b-6026-4653-ab79-e9c746993364"}]: dispatch 2023-10-01T19:40:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:35 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2272344b-6026-4653-ab79-e9c746993364"}]: dispatch 2023-10-01T19:40:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:35 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2272344b-6026-4653-ab79-e9c746993364"}]': finished 2023-10-01T19:40:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:35 smithi057 ceph-mon[114331]: osdmap e10: 2 total, 1 up, 2 in 2023-10-01T19:40:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:35 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:35 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3143630396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:40:36.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:36 smithi087 ceph-mon[109386]: pgmap v34: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:36.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:36 smithi057 ceph-mon[108286]: pgmap v34: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:36.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:36 smithi057 ceph-mon[114331]: pgmap v34: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:38.754 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:38 smithi057 ceph-mon[108286]: pgmap v35: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:38.755 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:38 smithi057 ceph-mon[114331]: pgmap v35: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:39.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:38 smithi087 ceph-mon[109386]: pgmap v35: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:39.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:39 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-01T19:40:39.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:39 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:39.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:39 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-01T19:40:39.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:39 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:40.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:39 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-01T19:40:40.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:39 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:40.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:40 smithi057 ceph-mon[108286]: pgmap v36: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:40.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:40 smithi057 ceph-mon[108286]: Deploying daemon osd.1 on smithi057 2023-10-01T19:40:40.830 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:40 smithi057 ceph-mon[114331]: pgmap v36: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:40.830 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:40 smithi057 ceph-mon[114331]: Deploying daemon osd.1 on smithi057 2023-10-01T19:40:41.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:40 smithi087 ceph-mon[109386]: pgmap v36: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:41.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:40 smithi087 ceph-mon[109386]: Deploying daemon osd.1 on smithi057 2023-10-01T19:40:42.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:42 smithi057 ceph-mon[108286]: pgmap v37: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:42.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:42 smithi057 ceph-mon[114331]: pgmap v37: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:43.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:42 smithi087 ceph-mon[109386]: pgmap v37: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:44.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:44 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:44.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:44 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:44.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:44 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:44.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:44 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:44.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:44 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:44.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:44 smithi087 ceph-mon[109386]: pgmap v38: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:44.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:44 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:44.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:44 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:44.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:44 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:44.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:44 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:44.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:44 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:44.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:44 smithi057 ceph-mon[108286]: pgmap v38: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:44.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:44 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:44.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:44 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:44.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:44 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:44.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:44 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:44.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:44 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:44.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:44 smithi057 ceph-mon[114331]: pgmap v38: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:45.446 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 1 on host 'smithi057' 2023-10-01T19:40:45.830 DEBUG:teuthology.orchestra.run.smithi057:osd.1> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.1.service 2023-10-01T19:40:45.833 INFO:tasks.cephadm:Deploying osd.2 on smithi057 with /dev/vg_nvme/lv_2... 2023-10-01T19:40:45.833 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-01T19:40:45.871 INFO:journalctl@ceph.osd.1.smithi057.stdout:-- Logs begin at Sun 2023-10-01 19:27:22 UTC. -- 2023-10-01T19:40:46.672 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-1[121553]: 2023-10-01T19:40:46.601+0000 7f616fbfa3c0 -1 osd.1 0 log_to_monitors true 2023-10-01T19:40:46.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:46.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:46.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:46.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:46.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:40:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:40:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[108286]: Adjusting osd_memory_target on smithi057 to 7535M 2023-10-01T19:40:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[108286]: pgmap v39: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:46.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:46.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:46.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:46.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:46.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:40:46.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:40:46.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[114331]: Adjusting osd_memory_target on smithi057 to 7535M 2023-10-01T19:40:46.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:46.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:46.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:46 smithi057 ceph-mon[114331]: pgmap v39: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:46.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:46 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:46.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:46 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:46.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:46 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:46.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:46 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:46.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:46 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:40:46.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:46 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:40:46.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:46 smithi087 ceph-mon[109386]: Adjusting osd_memory_target on smithi057 to 7535M 2023-10-01T19:40:46.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:46 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:46.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:46 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:46.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:46 smithi087 ceph-mon[109386]: pgmap v39: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:47.377 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:40:47] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:40:47.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[114331]: from='osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-01T19:40:47.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:47.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:47.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:47.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:47.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:47.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:47.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[108286]: from='osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-01T19:40:47.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:47 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:47.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:47 smithi087 ceph-mon[109386]: from='osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-01T19:40:47.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:47 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:47.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:47 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:47.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:47 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:47.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:47 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:47.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:47 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:47.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:47 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-01T19:40:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-01T19:40:48.199 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2023-10-01T19:40:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2023-10-01T19:40:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126842 s, 827 MB/s 2023-10-01T19:40:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2023-10-01T19:40:48.516 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:40:48] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:40:48.612 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi057:vg_nvme/lv_2 2023-10-01T19:40:48.866 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-1[121553]: 2023-10-01T19:40:48.527+0000 7f61665fd700 -1 osd.1 0 waiting for initial osdmap 2023-10-01T19:40:48.867 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-1[121553]: 2023-10-01T19:40:48.531+0000 7f6161795700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-01T19:40:48.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[108286]: Detected new or changed devices on smithi057 2023-10-01T19:40:48.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[108286]: from='osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-01T19:40:48.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[108286]: osdmap e11: 2 total, 1 up, 2 in 2023-10-01T19:40:48.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:48.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[108286]: from='osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:40:48.868 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[108286]: pgmap v41: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:48.868 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:48.868 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:48.868 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:48.868 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:48.868 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:48.868 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:48.868 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[114331]: Detected new or changed devices on smithi057 2023-10-01T19:40:48.868 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[114331]: from='osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-01T19:40:48.868 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[114331]: osdmap e11: 2 total, 1 up, 2 in 2023-10-01T19:40:48.868 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:48.869 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[114331]: from='osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:40:48.869 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[114331]: pgmap v41: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:48.869 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:48.869 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:48.869 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:48.869 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:48.869 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:48.869 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:48 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:48.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:48 smithi087 ceph-mon[109386]: Detected new or changed devices on smithi057 2023-10-01T19:40:48.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:48 smithi087 ceph-mon[109386]: from='osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-01T19:40:48.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:48 smithi087 ceph-mon[109386]: osdmap e11: 2 total, 1 up, 2 in 2023-10-01T19:40:48.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:48 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:48.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:48 smithi087 ceph-mon[109386]: from='osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:40:48.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:48 smithi087 ceph-mon[109386]: pgmap v41: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2023-10-01T19:40:48.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:48 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:48.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:48 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:48.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:48 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:40:48.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:48 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:48.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:48 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:40:48.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:48 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:49.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:49 smithi087 ceph-mon[109386]: from='osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-01T19:40:49.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:49 smithi087 ceph-mon[109386]: osdmap e12: 2 total, 1 up, 2 in 2023-10-01T19:40:49.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:49 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:49.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:49 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:49.911 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:49 smithi057 ceph-mon[108286]: from='osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-01T19:40:49.912 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:49 smithi057 ceph-mon[108286]: osdmap e12: 2 total, 1 up, 2 in 2023-10-01T19:40:49.912 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:49 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:49.912 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:49 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:49.912 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:49 smithi057 ceph-mon[114331]: from='osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-01T19:40:49.912 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:49 smithi057 ceph-mon[114331]: osdmap e12: 2 total, 1 up, 2 in 2023-10-01T19:40:49.912 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:49 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:49.912 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:49 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:50.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[114331]: purged_snaps scrub starts 2023-10-01T19:40:50.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[114331]: purged_snaps scrub ok 2023-10-01T19:40:50.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[114331]: osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730] boot 2023-10-01T19:40:50.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[114331]: osdmap e13: 2 total, 2 up, 2 in 2023-10-01T19:40:50.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:50.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[114331]: pgmap v44: 0 pgs: ; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:50.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:40:50.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:40:50.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:50.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[108286]: purged_snaps scrub starts 2023-10-01T19:40:50.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[108286]: purged_snaps scrub ok 2023-10-01T19:40:50.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[108286]: osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730] boot 2023-10-01T19:40:50.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[108286]: osdmap e13: 2 total, 2 up, 2 in 2023-10-01T19:40:50.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:50.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[108286]: pgmap v44: 0 pgs: ; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:50.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:40:50.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:40:50.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:50 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:50.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:50 smithi087 ceph-mon[109386]: purged_snaps scrub starts 2023-10-01T19:40:50.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:50 smithi087 ceph-mon[109386]: purged_snaps scrub ok 2023-10-01T19:40:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:50 smithi087 ceph-mon[109386]: osd.1 [v2:172.21.15.57:6810/3617577730,v1:172.21.15.57:6811/3617577730] boot 2023-10-01T19:40:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:50 smithi087 ceph-mon[109386]: osdmap e13: 2 total, 2 up, 2 in 2023-10-01T19:40:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:50 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:40:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:50 smithi087 ceph-mon[109386]: pgmap v44: 0 pgs: ; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:50 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:40:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:50 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:40:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:50 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:51.578 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:51 smithi087 ceph-mon[109386]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:40:51.578 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:51 smithi087 ceph-mon[109386]: osdmap e14: 2 total, 2 up, 2 in 2023-10-01T19:40:51.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:51 smithi057 ceph-mon[108286]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:40:51.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:51 smithi057 ceph-mon[108286]: osdmap e14: 2 total, 2 up, 2 in 2023-10-01T19:40:51.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:51 smithi057 ceph-mon[114331]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:40:51.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:51 smithi057 ceph-mon[114331]: osdmap e14: 2 total, 2 up, 2 in 2023-10-01T19:40:52.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:52 smithi087 ceph-mon[109386]: pgmap v46: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:52.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:52 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1910897776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63bc952e-d249-4ae3-a4cb-a50f415d9beb"}]: dispatch 2023-10-01T19:40:52.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:52 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1910897776' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63bc952e-d249-4ae3-a4cb-a50f415d9beb"}]': finished 2023-10-01T19:40:52.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:52 smithi087 ceph-mon[109386]: osdmap e15: 3 total, 2 up, 3 in 2023-10-01T19:40:52.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:52 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:40:52.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:52 smithi057 ceph-mon[108286]: pgmap v46: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:52.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:52 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1910897776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63bc952e-d249-4ae3-a4cb-a50f415d9beb"}]: dispatch 2023-10-01T19:40:52.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:52 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1910897776' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63bc952e-d249-4ae3-a4cb-a50f415d9beb"}]': finished 2023-10-01T19:40:52.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:52 smithi057 ceph-mon[108286]: osdmap e15: 3 total, 2 up, 3 in 2023-10-01T19:40:52.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:52 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:40:52.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:52 smithi057 ceph-mon[114331]: pgmap v46: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:52.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:52 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1910897776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63bc952e-d249-4ae3-a4cb-a50f415d9beb"}]: dispatch 2023-10-01T19:40:52.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:52 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1910897776' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63bc952e-d249-4ae3-a4cb-a50f415d9beb"}]': finished 2023-10-01T19:40:52.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:52 smithi057 ceph-mon[114331]: osdmap e15: 3 total, 2 up, 3 in 2023-10-01T19:40:52.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:52 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:40:53.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:40:52 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:40:52] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:40:53.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:53 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3592534819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:40:53.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:53 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:53.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:53 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:53.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:53 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3592534819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:40:53.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:53 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:53.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:53 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:53.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:53 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3592534819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:40:53.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:53 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:53.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:53 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:40:54.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:54 smithi087 ceph-mon[109386]: pgmap v48: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:54.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:54 smithi057 ceph-mon[108286]: pgmap v48: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:54.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:54 smithi057 ceph-mon[114331]: pgmap v48: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:56 smithi057 ceph-mon[108286]: pgmap v49: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:56.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:56 smithi057 ceph-mon[114331]: pgmap v49: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:57.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:56 smithi087 ceph-mon[109386]: pgmap v49: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:57.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-01T19:40:57.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:57.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-01T19:40:57.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:58.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-01T19:40:58.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:40:58.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:58 smithi057 ceph-mon[108286]: Deploying daemon osd.2 on smithi057 2023-10-01T19:40:58.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:40:58 smithi057 ceph-mon[108286]: pgmap v50: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:58.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:58 smithi057 ceph-mon[114331]: Deploying daemon osd.2 on smithi057 2023-10-01T19:40:58.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:40:58 smithi057 ceph-mon[114331]: pgmap v50: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:40:59.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:58 smithi087 ceph-mon[109386]: Deploying daemon osd.2 on smithi057 2023-10-01T19:40:59.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:40:58 smithi087 ceph-mon[109386]: pgmap v50: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:00.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:00 smithi057 ceph-mon[108286]: pgmap v51: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:00.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:00 smithi057 ceph-mon[114331]: pgmap v51: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:01.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:00 smithi087 ceph-mon[109386]: pgmap v51: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:02.494 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:02 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:02.494 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:02 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:02.494 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:02 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:02.494 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:02 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:02.494 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:02 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:02.494 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:02 smithi057 ceph-mon[114331]: pgmap v52: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:02.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:02 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:02.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:02 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:02.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:02 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:02.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:02 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:02.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:02 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:02.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:02 smithi057 ceph-mon[108286]: pgmap v52: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:02.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:02 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:02.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:02 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:02.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:02 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:02.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:02 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:02.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:02 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:02.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:02 smithi087 ceph-mon[109386]: pgmap v52: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:03.438 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 2 on host 'smithi057' 2023-10-01T19:41:03.873 DEBUG:teuthology.orchestra.run.smithi057:osd.2> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.2.service 2023-10-01T19:41:03.875 INFO:tasks.cephadm:Deploying osd.3 on smithi057 with /dev/vg_nvme/lv_1... 2023-10-01T19:41:03.876 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-01T19:41:03.914 INFO:journalctl@ceph.osd.2.smithi057.stdout:-- Logs begin at Sun 2023-10-01 19:27:22 UTC. -- 2023-10-01T19:41:04.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:04.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:04.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:04.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:04.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:04.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:04.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:04.586 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[108286]: Adjusting osd_memory_target on smithi057 to 5023M 2023-10-01T19:41:04.586 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:04.586 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:04.586 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[108286]: pgmap v53: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:04.586 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:04.586 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:04.586 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:04.586 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:04.586 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:04.587 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:04.587 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:04.587 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[114331]: Adjusting osd_memory_target on smithi057 to 5023M 2023-10-01T19:41:04.587 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:04.587 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:04.587 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-mon[114331]: pgmap v53: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:04.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:04.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:04.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:04 smithi087 ceph-mon[109386]: Adjusting osd_memory_target on smithi057 to 5023M 2023-10-01T19:41:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:04 smithi087 ceph-mon[109386]: pgmap v53: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:04.921 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 19:41:04 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-2[124933]: 2023-10-01T19:41:04.583+0000 7fafebe543c0 -1 osd.2 0 log_to_monitors true 2023-10-01T19:41:05.524 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-01T19:41:05.525 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-01T19:41:05.525 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2023-10-01T19:41:05.525 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2023-10-01T19:41:05.525 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133865 s, 783 MB/s 2023-10-01T19:41:05.525 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2023-10-01T19:41:05.877 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:05 smithi057 ceph-mon[108286]: from='osd.2 [v2:172.21.15.57:6818/2187972944,v1:172.21.15.57:6819/2187972944]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-01T19:41:05.877 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:05 smithi057 ceph-mon[108286]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-01T19:41:05.877 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:05 smithi057 ceph-mon[114331]: from='osd.2 [v2:172.21.15.57:6818/2187972944,v1:172.21.15.57:6819/2187972944]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-01T19:41:05.877 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:05 smithi057 ceph-mon[114331]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-01T19:41:05.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:05 smithi087 ceph-mon[109386]: from='osd.2 [v2:172.21.15.57:6818/2187972944,v1:172.21.15.57:6819/2187972944]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-01T19:41:05.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:05 smithi087 ceph-mon[109386]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-01T19:41:05.934 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi057:vg_nvme/lv_1 2023-10-01T19:41:06.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-mon[108286]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-01T19:41:06.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-mon[108286]: osdmap e16: 3 total, 2 up, 3 in 2023-10-01T19:41:06.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:06.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-mon[108286]: from='osd.2 [v2:172.21.15.57:6818/2187972944,v1:172.21.15.57:6819/2187972944]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:41:06.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-mon[108286]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:41:06.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-mon[108286]: pgmap v55: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:06.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:06 smithi087 ceph-mon[109386]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-01T19:41:06.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:06 smithi087 ceph-mon[109386]: osdmap e16: 3 total, 2 up, 3 in 2023-10-01T19:41:06.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:06 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:06.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:06 smithi087 ceph-mon[109386]: from='osd.2 [v2:172.21.15.57:6818/2187972944,v1:172.21.15.57:6819/2187972944]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:41:06.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:06 smithi087 ceph-mon[109386]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:41:06.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:06 smithi087 ceph-mon[109386]: pgmap v55: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:06.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-mon[114331]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-01T19:41:06.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-mon[114331]: osdmap e16: 3 total, 2 up, 3 in 2023-10-01T19:41:06.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:06.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-mon[114331]: from='osd.2 [v2:172.21.15.57:6818/2187972944,v1:172.21.15.57:6819/2187972944]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:41:06.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-mon[114331]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:41:06.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-mon[114331]: pgmap v55: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:06.922 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-2[124933]: 2023-10-01T19:41:06.511+0000 7fafe2857700 -1 osd.2 0 waiting for initial osdmap 2023-10-01T19:41:06.922 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 19:41:06 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-2[124933]: 2023-10-01T19:41:06.515+0000 7fafde1f0700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-01T19:41:07.637 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:07 smithi057 ceph-mon[108286]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-01T19:41:07.637 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:07 smithi057 ceph-mon[108286]: osdmap e17: 3 total, 2 up, 3 in 2023-10-01T19:41:07.637 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:07 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:07.637 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:07 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:07.638 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:07 smithi057 ceph-mon[114331]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-01T19:41:07.638 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:07 smithi057 ceph-mon[114331]: osdmap e17: 3 total, 2 up, 3 in 2023-10-01T19:41:07.638 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:07 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:07.638 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:07 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:07.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:07 smithi087 ceph-mon[109386]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-01T19:41:07.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:07 smithi087 ceph-mon[109386]: osdmap e17: 3 total, 2 up, 3 in 2023-10-01T19:41:07.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:07 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:07.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:07 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:08.459 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:41:08] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:41:08.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: purged_snaps scrub starts 2023-10-01T19:41:08.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: purged_snaps scrub ok 2023-10-01T19:41:08.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:08.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: osd.2 [v2:172.21.15.57:6818/2187972944,v1:172.21.15.57:6819/2187972944] boot 2023-10-01T19:41:08.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: osdmap e18: 3 total, 3 up, 3 in 2023-10-01T19:41:08.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:08.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: pgmap v58: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:08.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2023-10-01T19:41:08.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:41:08.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:41:08.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:08.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:08.792 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:08.792 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:08.792 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:08.792 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:08.792 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:08.792 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: purged_snaps scrub starts 2023-10-01T19:41:08.792 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: purged_snaps scrub ok 2023-10-01T19:41:08.792 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:08.792 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: osd.2 [v2:172.21.15.57:6818/2187972944,v1:172.21.15.57:6819/2187972944] boot 2023-10-01T19:41:08.792 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: osdmap e18: 3 total, 3 up, 3 in 2023-10-01T19:41:08.792 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:08.793 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: pgmap v58: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:08.793 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2023-10-01T19:41:08.793 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:41:08.793 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:41:08.793 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:08.793 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:08.793 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:08.793 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:08.793 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:08.793 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:08.793 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:08.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: purged_snaps scrub starts 2023-10-01T19:41:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: purged_snaps scrub ok 2023-10-01T19:41:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: osd.2 [v2:172.21.15.57:6818/2187972944,v1:172.21.15.57:6819/2187972944] boot 2023-10-01T19:41:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: osdmap e18: 3 total, 3 up, 3 in 2023-10-01T19:41:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:41:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: pgmap v58: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2023-10-01T19:41:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:41:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:41:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:08.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:08.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:08.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:08.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:09.581 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[114331]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:41:09.581 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[114331]: Detected new or changed devices on smithi057 2023-10-01T19:41:09.581 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[114331]: pgmap v59: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:09.581 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[114331]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2023-10-01T19:41:09.581 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2023-10-01T19:41:09.581 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[114331]: osdmap e19: 3 total, 3 up, 3 in 2023-10-01T19:41:09.581 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-01T19:41:09.581 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[108286]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:41:09.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[108286]: Detected new or changed devices on smithi057 2023-10-01T19:41:09.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[108286]: pgmap v59: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:09.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[108286]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2023-10-01T19:41:09.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2023-10-01T19:41:09.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[108286]: osdmap e19: 3 total, 3 up, 3 in 2023-10-01T19:41:09.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-01T19:41:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:09 smithi087 ceph-mon[109386]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:41:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:09 smithi087 ceph-mon[109386]: Detected new or changed devices on smithi057 2023-10-01T19:41:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:09 smithi087 ceph-mon[109386]: pgmap v59: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-10-01T19:41:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:09 smithi087 ceph-mon[109386]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2023-10-01T19:41:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2023-10-01T19:41:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:09 smithi087 ceph-mon[109386]: osdmap e19: 3 total, 3 up, 3 in 2023-10-01T19:41:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:09 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-01T19:41:09.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:41:09 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:41:09] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:41:10.421 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 19:41:10 smithi057 sudo[126766]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-01T19:41:10.421 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 19:41:10 smithi057 sudo[126766]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-01T19:41:10.421 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 19:41:10 smithi057 sudo[126766]: pam_unix(sudo:session): session closed for user root 2023-10-01T19:41:10.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-01T19:41:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: osdmap e20: 3 total, 3 up, 3 in 2023-10-01T19:41:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: Detected new or changed devices on smithi057 2023-10-01T19:41:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: pgmap v62: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3026368007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "039ec24f-b99e-4248-b0d9-b9e61bfd8833"}]: dispatch 2023-10-01T19:41:10.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "039ec24f-b99e-4248-b0d9-b9e61bfd8833"}]: dispatch 2023-10-01T19:41:10.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "039ec24f-b99e-4248-b0d9-b9e61bfd8833"}]': finished 2023-10-01T19:41:10.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: osdmap e21: 4 total, 3 up, 4 in 2023-10-01T19:41:10.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:10.921 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 19:41:10 smithi057 sudo[126780]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-01T19:41:10.921 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 19:41:10 smithi057 sudo[126780]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-01T19:41:10.921 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 19:41:10 smithi057 sudo[126780]: pam_unix(sudo:session): session closed for user root 2023-10-01T19:41:10.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-01T19:41:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: osdmap e20: 3 total, 3 up, 3 in 2023-10-01T19:41:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: Detected new or changed devices on smithi057 2023-10-01T19:41:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: pgmap v62: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3026368007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "039ec24f-b99e-4248-b0d9-b9e61bfd8833"}]: dispatch 2023-10-01T19:41:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "039ec24f-b99e-4248-b0d9-b9e61bfd8833"}]: dispatch 2023-10-01T19:41:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "039ec24f-b99e-4248-b0d9-b9e61bfd8833"}]': finished 2023-10-01T19:41:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: osdmap e21: 4 total, 3 up, 4 in 2023-10-01T19:41:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:10.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-01T19:41:10.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: osdmap e20: 3 total, 3 up, 3 in 2023-10-01T19:41:10.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:10.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: Detected new or changed devices on smithi057 2023-10-01T19:41:10.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:10.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:10.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:10.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:10.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: pgmap v62: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:10.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:10.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3026368007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "039ec24f-b99e-4248-b0d9-b9e61bfd8833"}]: dispatch 2023-10-01T19:41:10.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "039ec24f-b99e-4248-b0d9-b9e61bfd8833"}]: dispatch 2023-10-01T19:41:10.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "039ec24f-b99e-4248-b0d9-b9e61bfd8833"}]': finished 2023-10-01T19:41:10.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: osdmap e21: 4 total, 3 up, 4 in 2023-10-01T19:41:10.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:11.421 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 19:41:11 smithi057 sudo[126854]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-01T19:41:11.421 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 19:41:11 smithi057 sudo[126854]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-01T19:41:11.421 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 19:41:11 smithi057 sudo[126854]: pam_unix(sudo:session): session closed for user root 2023-10-01T19:41:11.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:11 smithi087 ceph-mon[109386]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2023-10-01T19:41:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:11 smithi087 ceph-mon[109386]: Cluster is now healthy 2023-10-01T19:41:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:11 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:11 smithi087 ceph-mon[109386]: mgrmap e15: y(active, since 103s), standbys: x 2023-10-01T19:41:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:11 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1952081153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:41:11.921 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 19:41:11 smithi057 sudo[126859]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-01T19:41:11.921 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 19:41:11 smithi057 sudo[126859]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-01T19:41:11.921 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 19:41:11 smithi057 sudo[126859]: pam_unix(sudo:session): session closed for user root 2023-10-01T19:41:11.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:11 smithi057 ceph-mon[108286]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2023-10-01T19:41:11.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:11 smithi057 ceph-mon[108286]: Cluster is now healthy 2023-10-01T19:41:11.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:11 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:11.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:11 smithi057 ceph-mon[108286]: mgrmap e15: y(active, since 103s), standbys: x 2023-10-01T19:41:11.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:11 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1952081153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:41:11.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:11 smithi057 ceph-mon[114331]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2023-10-01T19:41:11.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:11 smithi057 ceph-mon[114331]: Cluster is now healthy 2023-10-01T19:41:11.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:11 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:11.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:11 smithi057 ceph-mon[114331]: mgrmap e15: y(active, since 103s), standbys: x 2023-10-01T19:41:11.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:11 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1952081153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:41:12.421 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 19:41:12 smithi057 sudo[127233]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-01T19:41:12.421 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 19:41:12 smithi057 sudo[127233]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-01T19:41:12.421 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 19:41:12 smithi057 sudo[127233]: pam_unix(sudo:session): session closed for user root 2023-10-01T19:41:12.921 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 19:41:12 smithi057 sudo[127425]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-01T19:41:12.921 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 19:41:12 smithi057 sudo[127425]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-01T19:41:12.921 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 19:41:12 smithi057 sudo[127425]: pam_unix(sudo:session): session closed for user root 2023-10-01T19:41:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:12 smithi057 sudo[127623]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-01T19:41:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:12 smithi057 sudo[127623]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-01T19:41:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:12 smithi057 sudo[127623]: pam_unix(sudo:session): session closed for user root 2023-10-01T19:41:13.541 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:13 smithi057 sudo[127658]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-01T19:41:13.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:13 smithi057 sudo[127658]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-01T19:41:13.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:13 smithi057 sudo[127658]: pam_unix(sudo:session): session closed for user root 2023-10-01T19:41:13.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:13 smithi057 ceph-mon[108286]: pgmap v64: 1 pgs: 1 creating+peering; 0 B data, 16 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:13.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:13 smithi087 ceph-mon[109386]: pgmap v64: 1 pgs: 1 creating+peering; 0 B data, 16 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:14.207 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:13 smithi057 sudo[127741]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-01T19:41:14.207 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:13 smithi057 sudo[127741]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-01T19:41:14.207 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:13 smithi057 sudo[127741]: pam_unix(sudo:session): session closed for user root 2023-10-01T19:41:14.525 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:41:14] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:41:14.525 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 sudo[127792]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-01T19:41:14.525 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 sudo[127792]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-01T19:41:14.526 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 sudo[127792]: pam_unix(sudo:session): session closed for user root 2023-10-01T19:41:14.526 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: pgmap v64: 1 pgs: 1 creating+peering; 0 B data, 16 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:14.876 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 sudo[112352]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-01T19:41:14.876 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 sudo[112352]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-01T19:41:14.876 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 sudo[112352]: pam_unix(sudo:session): session closed for user root 2023-10-01T19:41:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 sudo[112355]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-01T19:41:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 sudo[112355]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-01T19:41:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 sudo[112355]: pam_unix(sudo:session): session closed for user root 2023-10-01T19:41:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-01T19:41:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-01T19:41:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:41:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:41:15.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:41:15.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-01T19:41:15.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:41:15.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:15.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:41:15.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:41:15.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: pgmap v65: 1 pgs: 1 creating+peering; 0 B data, 17 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:15.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-01T19:41:15.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:41:15.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:41:15.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:41:15.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:14 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-01T19:41:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-01T19:41:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:41:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:41:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:41:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-01T19:41:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:41:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:41:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:41:15.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: pgmap v65: 1 pgs: 1 creating+peering; 0 B data, 17 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:15.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-01T19:41:15.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:41:15.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:41:15.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:41:15.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:15.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-01T19:41:15.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-01T19:41:15.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:41:15.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:41:15.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:41:15.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-01T19:41:15.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:41:15.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:15.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:41:15.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:41:15.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: pgmap v65: 1 pgs: 1 creating+peering; 0 B data, 17 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:15.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-01T19:41:15.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:41:15.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:41:15.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:41:15.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:14 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:16.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:15 smithi057 ceph-mon[108286]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-01T19:41:16.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:15 smithi057 ceph-mon[108286]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-01T19:41:16.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:15 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:16.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:15 smithi057 ceph-mon[108286]: pgmap v66: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:16.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:15 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-01T19:41:16.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:15 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:15 smithi057 ceph-mon[114331]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-01T19:41:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:15 smithi057 ceph-mon[114331]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-01T19:41:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:15 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:15 smithi057 ceph-mon[114331]: pgmap v66: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:15 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-01T19:41:16.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:15 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:16.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:15 smithi087 ceph-mon[109386]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-01T19:41:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:15 smithi087 ceph-mon[109386]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-01T19:41:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:15 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:15 smithi087 ceph-mon[109386]: pgmap v66: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:15 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-01T19:41:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:15 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:17.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:16 smithi057 ceph-mon[108286]: Deploying daemon osd.3 on smithi057 2023-10-01T19:41:17.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:16 smithi057 ceph-mon[114331]: Deploying daemon osd.3 on smithi057 2023-10-01T19:41:17.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:16 smithi087 ceph-mon[109386]: Deploying daemon osd.3 on smithi057 2023-10-01T19:41:18.169 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:17 smithi057 ceph-mon[108286]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:18.169 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:17 smithi057 ceph-mon[114331]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:18.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:17 smithi087 ceph-mon[109386]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2023-10-01T19:41:20.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:20 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:20.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:20 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:20.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:20 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:20.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:20 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:20.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:20 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:20.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:20 smithi057 ceph-mon[108286]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2023-10-01T19:41:20.624 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:20 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:20.624 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:20 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:20.624 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:20 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:20.624 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:20 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:20.624 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:20 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:20.625 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:20 smithi057 ceph-mon[114331]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2023-10-01T19:41:20.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:20 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:20.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:20 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:20.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:20 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:20.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:20 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:20.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:20 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:20.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:20 smithi087 ceph-mon[109386]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2023-10-01T19:41:21.574 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 3 on host 'smithi057' 2023-10-01T19:41:21.947 DEBUG:teuthology.orchestra.run.smithi057:osd.3> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.3.service 2023-10-01T19:41:21.950 INFO:tasks.cephadm:Deploying osd.4 on smithi087 with /dev/vg_nvme/lv_4... 2023-10-01T19:41:21.950 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-01T19:41:21.987 INFO:journalctl@ceph.osd.3.smithi057.stdout:-- Logs begin at Sun 2023-10-01 19:27:22 UTC. -- 2023-10-01T19:41:22.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:22 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:22.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:22 smithi087 ceph-mon[109386]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-10-01T19:41:22.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:22 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:22.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:22 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:22.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:22 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:22.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:22 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:22.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:22 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:22.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:22 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:22.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:22 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:22.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:22 smithi087 ceph-mon[109386]: Adjusting osd_memory_target on smithi057 to 3767M 2023-10-01T19:41:22.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:22 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:22.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:22 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:22.921 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-3[128379]: 2023-10-01T19:41:22.639+0000 7f3eca4e23c0 -1 osd.3 0 log_to_monitors true 2023-10-01T19:41:22.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:22.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[114331]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-10-01T19:41:22.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[114331]: Adjusting osd_memory_target on smithi057 to 3767M 2023-10-01T19:41:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:22.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[108286]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-10-01T19:41:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[108286]: Adjusting osd_memory_target on smithi057 to 3767M 2023-10-01T19:41:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:22 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:23.323 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-01T19:41:23.323 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-01T19:41:23.323 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2023-10-01T19:41:23.323 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2023-10-01T19:41:23.324 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113847 s, 921 MB/s 2023-10-01T19:41:23.324 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2023-10-01T19:41:23.609 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:23 smithi087 ceph-mon[109386]: from='osd.3 [v2:172.21.15.57:6826/1040473437,v1:172.21.15.57:6827/1040473437]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-01T19:41:23.610 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:23 smithi087 ceph-mon[109386]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-01T19:41:23.653 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi087:vg_nvme/lv_4 2023-10-01T19:41:23.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:23 smithi057 ceph-mon[108286]: from='osd.3 [v2:172.21.15.57:6826/1040473437,v1:172.21.15.57:6827/1040473437]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-01T19:41:23.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:23 smithi057 ceph-mon[108286]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-01T19:41:23.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:23 smithi057 ceph-mon[114331]: from='osd.3 [v2:172.21.15.57:6826/1040473437,v1:172.21.15.57:6827/1040473437]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-01T19:41:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:23 smithi057 ceph-mon[114331]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-01T19:41:24.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:24 smithi087 ceph-mon[109386]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-10-01T19:41:24.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:24 smithi087 ceph-mon[109386]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-01T19:41:24.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:24 smithi087 ceph-mon[109386]: osdmap e22: 4 total, 3 up, 4 in 2023-10-01T19:41:24.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:24 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:24.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:24 smithi087 ceph-mon[109386]: from='osd.3 [v2:172.21.15.57:6826/1040473437,v1:172.21.15.57:6827/1040473437]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:41:24.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:24 smithi087 ceph-mon[109386]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:41:24.921 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-3[128379]: 2023-10-01T19:41:24.637+0000 7f3ec0ee5700 -1 osd.3 0 waiting for initial osdmap 2023-10-01T19:41:24.921 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-3[128379]: 2023-10-01T19:41:24.641+0000 7f3ebb87c700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-01T19:41:24.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-mon[114331]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-10-01T19:41:24.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-mon[114331]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-01T19:41:24.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-mon[114331]: osdmap e22: 4 total, 3 up, 4 in 2023-10-01T19:41:24.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:24.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-mon[114331]: from='osd.3 [v2:172.21.15.57:6826/1040473437,v1:172.21.15.57:6827/1040473437]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:41:24.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-mon[114331]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:41:24.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-mon[108286]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-10-01T19:41:24.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-mon[108286]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-01T19:41:24.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-mon[108286]: osdmap e22: 4 total, 3 up, 4 in 2023-10-01T19:41:24.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:24.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-mon[108286]: from='osd.3 [v2:172.21.15.57:6826/1040473437,v1:172.21.15.57:6827/1040473437]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:41:24.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:24 smithi057 ceph-mon[108286]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T19:41:25.849 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:25 smithi087 ceph-mon[109386]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-01T19:41:25.849 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:25 smithi087 ceph-mon[109386]: osdmap e23: 4 total, 3 up, 4 in 2023-10-01T19:41:25.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:25 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:25.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:25 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:25.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:25 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:41:25.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:25 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:41:25.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:25 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:25.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[108286]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-01T19:41:25.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[108286]: osdmap e23: 4 total, 3 up, 4 in 2023-10-01T19:41:25.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:25.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:25.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:41:25.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:41:25.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:25.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[114331]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-01T19:41:25.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[114331]: osdmap e23: 4 total, 3 up, 4 in 2023-10-01T19:41:25.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:25.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:41:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:41:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:25 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:26.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: purged_snaps scrub starts 2023-10-01T19:41:26.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: purged_snaps scrub ok 2023-10-01T19:41:26.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:41:26.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-10-01T19:41:26.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:26.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: osd.3 [v2:172.21.15.57:6826/1040473437,v1:172.21.15.57:6827/1040473437] boot 2023-10-01T19:41:26.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: osdmap e24: 4 total, 4 up, 4 in 2023-10-01T19:41:26.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:26.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: from='client.? 172.21.15.87:0/3969393833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "709bab5f-07ff-4bf9-9c9c-942bbc4901a9"}]: dispatch 2023-10-01T19:41:26.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "709bab5f-07ff-4bf9-9c9c-942bbc4901a9"}]: dispatch 2023-10-01T19:41:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "709bab5f-07ff-4bf9-9c9c-942bbc4901a9"}]': finished 2023-10-01T19:41:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: osdmap e25: 5 total, 4 up, 5 in 2023-10-01T19:41:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:26.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: purged_snaps scrub starts 2023-10-01T19:41:26.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: purged_snaps scrub ok 2023-10-01T19:41:26.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:41:26.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-10-01T19:41:26.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:26.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: osd.3 [v2:172.21.15.57:6826/1040473437,v1:172.21.15.57:6827/1040473437] boot 2023-10-01T19:41:26.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: osdmap e24: 4 total, 4 up, 4 in 2023-10-01T19:41:26.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:26.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: from='client.? 172.21.15.87:0/3969393833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "709bab5f-07ff-4bf9-9c9c-942bbc4901a9"}]: dispatch 2023-10-01T19:41:26.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "709bab5f-07ff-4bf9-9c9c-942bbc4901a9"}]: dispatch 2023-10-01T19:41:26.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "709bab5f-07ff-4bf9-9c9c-942bbc4901a9"}]': finished 2023-10-01T19:41:26.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: osdmap e25: 5 total, 4 up, 5 in 2023-10-01T19:41:26.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: purged_snaps scrub starts 2023-10-01T19:41:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: purged_snaps scrub ok 2023-10-01T19:41:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:41:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-10-01T19:41:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: osd.3 [v2:172.21.15.57:6826/1040473437,v1:172.21.15.57:6827/1040473437] boot 2023-10-01T19:41:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: osdmap e24: 4 total, 4 up, 4 in 2023-10-01T19:41:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:41:27.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: from='client.? 172.21.15.87:0/3969393833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "709bab5f-07ff-4bf9-9c9c-942bbc4901a9"}]: dispatch 2023-10-01T19:41:27.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "709bab5f-07ff-4bf9-9c9c-942bbc4901a9"}]: dispatch 2023-10-01T19:41:27.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "709bab5f-07ff-4bf9-9c9c-942bbc4901a9"}]': finished 2023-10-01T19:41:27.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: osdmap e25: 5 total, 4 up, 5 in 2023-10-01T19:41:27.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:27.732 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:27 smithi057 ceph-mon[108286]: from='client.? 172.21.15.87:0/2829234123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:41:27.732 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:27 smithi057 ceph-mon[114331]: from='client.? 172.21.15.87:0/2829234123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:41:28.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:27 smithi087 ceph-mon[109386]: from='client.? 172.21.15.87:0/2829234123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:41:28.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:28 smithi057 ceph-mon[108286]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:28.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:41:28.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:41:28.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:28 smithi057 ceph-mon[114331]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:28.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:28 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:41:28.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:28 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:41:29.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:28 smithi087 ceph-mon[109386]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:29.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:28 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:41:29.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:28 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:41:29.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:41:29 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:41:29] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:41:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:30 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:30 smithi087 ceph-mon[109386]: Detected new or changed devices on smithi057 2023-10-01T19:41:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:30 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:30 smithi087 ceph-mon[109386]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:30.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:30 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:30.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:30 smithi057 ceph-mon[108286]: Detected new or changed devices on smithi057 2023-10-01T19:41:30.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:30 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:30.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:30 smithi057 ceph-mon[108286]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:30.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:30 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:30.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:30 smithi057 ceph-mon[114331]: Detected new or changed devices on smithi057 2023-10-01T19:41:30.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:30 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:30.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:30 smithi057 ceph-mon[114331]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:31.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:31 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-01T19:41:31.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:31 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:31.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:31 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-01T19:41:31.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:31 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:31.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:31 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-01T19:41:31.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:31 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:32.794 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:32 smithi087 ceph-mon[109386]: Deploying daemon osd.4 on smithi087 2023-10-01T19:41:32.794 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:32 smithi087 ceph-mon[109386]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:32.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:32 smithi057 ceph-mon[108286]: Deploying daemon osd.4 on smithi087 2023-10-01T19:41:32.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:32 smithi057 ceph-mon[108286]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:32.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:32 smithi057 ceph-mon[114331]: Deploying daemon osd.4 on smithi087 2023-10-01T19:41:32.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:32 smithi057 ceph-mon[114331]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:34.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:34 smithi087 ceph-mon[109386]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:34.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:34 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:34.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:34 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:34.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:41:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:41:34] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:41:34.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:34 smithi057 ceph-mon[108286]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:34.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:34 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:34.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:34 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:34.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:34 smithi057 ceph-mon[114331]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:34.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:34 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:34.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:34 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:35.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:41:35 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:41:35] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:41:36.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:36 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:36.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:36 smithi087 ceph-mon[109386]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:36 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:36 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:36 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:36 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:36 smithi087 ceph-mon[109386]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:36 smithi087 ceph-mon[109386]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:36 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:36 smithi087 ceph-mon[109386]: Reconfiguring osd.4 (unknown last config time)... 2023-10-01T19:41:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:36 smithi087 ceph-mon[109386]: Reconfiguring daemon osd.4 on smithi087 2023-10-01T19:41:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:36 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:36.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:36.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[114331]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:36.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:36.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:36.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:36.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:36.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[114331]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:36.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[114331]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:36.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:36.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[114331]: Reconfiguring osd.4 (unknown last config time)... 2023-10-01T19:41:36.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[114331]: Reconfiguring daemon osd.4 on smithi087 2023-10-01T19:41:36.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:36.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:36.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[108286]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:36.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:36.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:36.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:36.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:36.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[108286]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:36.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[108286]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:36.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:36.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[108286]: Reconfiguring osd.4 (unknown last config time)... 2023-10-01T19:41:36.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[108286]: Reconfiguring daemon osd.4 on smithi087 2023-10-01T19:41:36.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:36 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:37.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:37 smithi087 ceph-mon[109386]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-10-01T19:41:37.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:37 smithi057 ceph-mon[108286]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-10-01T19:41:37.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:37 smithi057 ceph-mon[114331]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-10-01T19:41:38.226 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 4 on host 'smithi087' 2023-10-01T19:41:38.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:38.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:38.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:38.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:38.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:38.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:38.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:38.507 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:38.507 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:38.507 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2023-10-01T19:41:38.507 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: Cluster is now healthy 2023-10-01T19:41:38.507 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.507 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.591 DEBUG:teuthology.orchestra.run.smithi087:osd.4> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.4.service 2023-10-01T19:41:38.594 INFO:tasks.cephadm:Deploying osd.5 on smithi087 with /dev/vg_nvme/lv_3... 2023-10-01T19:41:38.594 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-01T19:41:38.624 INFO:journalctl@ceph.osd.4.smithi087.stdout:-- Logs begin at Sun 2023-10-01 19:27:12 UTC. -- 2023-10-01T19:41:38.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:38.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:38.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2023-10-01T19:41:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: Cluster is now healthy 2023-10-01T19:41:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:38.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:38.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:38.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:38.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:38.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:38.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:38.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:38.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:38.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2023-10-01T19:41:38.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: Cluster is now healthy 2023-10-01T19:41:38.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:38.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:39.563 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:39 smithi087 ceph-mon[109386]: Adjusting osd_memory_target on smithi087 to 16089M 2023-10-01T19:41:39.563 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:39 smithi087 ceph-mon[109386]: from='osd.4 [v2:172.21.15.87:6800/2800415383,v1:172.21.15.87:6801/2800415383]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-01T19:41:39.564 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:39 smithi087 ceph-mon[109386]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-01T19:41:39.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:39 smithi057 ceph-mon[108286]: Adjusting osd_memory_target on smithi087 to 16089M 2023-10-01T19:41:39.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:39 smithi057 ceph-mon[108286]: from='osd.4 [v2:172.21.15.87:6800/2800415383,v1:172.21.15.87:6801/2800415383]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-01T19:41:39.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:39 smithi057 ceph-mon[108286]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-01T19:41:39.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:39 smithi057 ceph-mon[114331]: Adjusting osd_memory_target on smithi087 to 16089M 2023-10-01T19:41:39.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:39 smithi057 ceph-mon[114331]: from='osd.4 [v2:172.21.15.87:6800/2800415383,v1:172.21.15.87:6801/2800415383]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-01T19:41:39.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:39 smithi057 ceph-mon[114331]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-01T19:41:39.991 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-01T19:41:39.991 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-01T19:41:39.991 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2023-10-01T19:41:39.991 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2023-10-01T19:41:39.991 INFO:teuthology.orchestra.run.smithi087.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114882 s, 913 MB/s 2023-10-01T19:41:39.991 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2023-10-01T19:41:40.359 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi087:vg_nvme/lv_3 2023-10-01T19:41:40.632 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 19:41:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T19:41:40.326+0000 7fa3b8fb5700 -1 osd.4 0 waiting for initial osdmap 2023-10-01T19:41:40.633 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 19:41:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T19:41:40.330+0000 7fa3b414d700 -1 osd.4 27 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-01T19:41:40.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:40 smithi087 ceph-mon[109386]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:40.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:40 smithi087 ceph-mon[109386]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-01T19:41:40.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:40 smithi087 ceph-mon[109386]: osdmap e26: 5 total, 4 up, 5 in 2023-10-01T19:41:40.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:40 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:40.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:40 smithi087 ceph-mon[109386]: from='osd.4 [v2:172.21.15.87:6800/2800415383,v1:172.21.15.87:6801/2800415383]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:41:40.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:40 smithi087 ceph-mon[109386]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:41:40.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:40 smithi057 ceph-mon[108286]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:40.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:40 smithi057 ceph-mon[108286]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-01T19:41:40.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:40 smithi057 ceph-mon[108286]: osdmap e26: 5 total, 4 up, 5 in 2023-10-01T19:41:40.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:40 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:40.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:40 smithi057 ceph-mon[108286]: from='osd.4 [v2:172.21.15.87:6800/2800415383,v1:172.21.15.87:6801/2800415383]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:41:40.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:40 smithi057 ceph-mon[108286]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:41:40.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:40 smithi057 ceph-mon[114331]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:40.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:40 smithi057 ceph-mon[114331]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-01T19:41:40.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:40 smithi057 ceph-mon[114331]: osdmap e26: 5 total, 4 up, 5 in 2023-10-01T19:41:40.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:40 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:40.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:40 smithi057 ceph-mon[114331]: from='osd.4 [v2:172.21.15.87:6800/2800415383,v1:172.21.15.87:6801/2800415383]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:41:40.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:40 smithi057 ceph-mon[114331]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:41:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:41 smithi087 ceph-mon[109386]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-10-01T19:41:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:41 smithi087 ceph-mon[109386]: osdmap e27: 5 total, 4 up, 5 in 2023-10-01T19:41:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:41 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:41 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:41.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:41 smithi057 ceph-mon[108286]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-10-01T19:41:41.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:41 smithi057 ceph-mon[108286]: osdmap e27: 5 total, 4 up, 5 in 2023-10-01T19:41:41.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:41 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:41.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:41 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:41.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:41 smithi057 ceph-mon[114331]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-10-01T19:41:41.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:41 smithi057 ceph-mon[114331]: osdmap e27: 5 total, 4 up, 5 in 2023-10-01T19:41:41.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:41 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:41.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:41 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:42.612 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:42 smithi087 ceph-mon[109386]: purged_snaps scrub starts 2023-10-01T19:41:42.612 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:42 smithi087 ceph-mon[109386]: purged_snaps scrub ok 2023-10-01T19:41:42.612 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:42 smithi087 ceph-mon[109386]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:42.612 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:42 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:42.612 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:42 smithi087 ceph-mon[109386]: osd.4 [v2:172.21.15.87:6800/2800415383,v1:172.21.15.87:6801/2800415383] boot 2023-10-01T19:41:42.612 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:42 smithi087 ceph-mon[109386]: osdmap e28: 5 total, 5 up, 5 in 2023-10-01T19:41:42.613 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:42 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:42.613 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:42 smithi087 ceph-mon[109386]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:41:42.613 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:42 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:41:42.613 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:42 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:41:42.613 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:42 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:42.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[114331]: purged_snaps scrub starts 2023-10-01T19:41:42.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[114331]: purged_snaps scrub ok 2023-10-01T19:41:42.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[114331]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[114331]: osd.4 [v2:172.21.15.87:6800/2800415383,v1:172.21.15.87:6801/2800415383] boot 2023-10-01T19:41:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[114331]: osdmap e28: 5 total, 5 up, 5 in 2023-10-01T19:41:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[114331]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:41:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:41:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:41:42.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[108286]: purged_snaps scrub starts 2023-10-01T19:41:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[108286]: purged_snaps scrub ok 2023-10-01T19:41:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[108286]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-10-01T19:41:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[108286]: osd.4 [v2:172.21.15.87:6800/2800415383,v1:172.21.15.87:6801/2800415383] boot 2023-10-01T19:41:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[108286]: osdmap e28: 5 total, 5 up, 5 in 2023-10-01T19:41:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:41:42.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[108286]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:41:42.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:41:42.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:41:42.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:42 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:43 smithi087 ceph-mon[109386]: osdmap e29: 5 total, 5 up, 5 in 2023-10-01T19:41:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:43 smithi087 ceph-mon[109386]: from='client.? 172.21.15.87:0/1723077926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "839f46cd-0e7c-4f51-b784-3c85cc3dde2f"}]: dispatch 2023-10-01T19:41:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:43 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "839f46cd-0e7c-4f51-b784-3c85cc3dde2f"}]: dispatch 2023-10-01T19:41:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:43 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "839f46cd-0e7c-4f51-b784-3c85cc3dde2f"}]': finished 2023-10-01T19:41:43.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:43 smithi087 ceph-mon[109386]: osdmap e30: 6 total, 5 up, 6 in 2023-10-01T19:41:43.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:43 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:43.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:43 smithi057 ceph-mon[108286]: osdmap e29: 5 total, 5 up, 5 in 2023-10-01T19:41:43.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:43 smithi057 ceph-mon[108286]: from='client.? 172.21.15.87:0/1723077926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "839f46cd-0e7c-4f51-b784-3c85cc3dde2f"}]: dispatch 2023-10-01T19:41:43.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:43 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "839f46cd-0e7c-4f51-b784-3c85cc3dde2f"}]: dispatch 2023-10-01T19:41:43.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:43 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "839f46cd-0e7c-4f51-b784-3c85cc3dde2f"}]': finished 2023-10-01T19:41:43.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:43 smithi057 ceph-mon[108286]: osdmap e30: 6 total, 5 up, 6 in 2023-10-01T19:41:43.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:43 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:43.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:43 smithi057 ceph-mon[114331]: osdmap e29: 5 total, 5 up, 5 in 2023-10-01T19:41:43.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:43 smithi057 ceph-mon[114331]: from='client.? 172.21.15.87:0/1723077926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "839f46cd-0e7c-4f51-b784-3c85cc3dde2f"}]: dispatch 2023-10-01T19:41:43.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:43 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "839f46cd-0e7c-4f51-b784-3c85cc3dde2f"}]: dispatch 2023-10-01T19:41:43.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:43 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "839f46cd-0e7c-4f51-b784-3c85cc3dde2f"}]': finished 2023-10-01T19:41:43.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:43 smithi057 ceph-mon[114331]: osdmap e30: 6 total, 5 up, 6 in 2023-10-01T19:41:43.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:43 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:44.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:44 smithi087 ceph-mon[109386]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:44.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:44 smithi087 ceph-mon[109386]: from='client.? 172.21.15.87:0/178474798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:41:44.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:44 smithi087 ceph-mon[109386]: osdmap e31: 6 total, 5 up, 6 in 2023-10-01T19:41:44.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:44 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:44.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:44 smithi057 ceph-mon[108286]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:44.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:44 smithi057 ceph-mon[108286]: from='client.? 172.21.15.87:0/178474798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:41:44.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:44 smithi057 ceph-mon[108286]: osdmap e31: 6 total, 5 up, 6 in 2023-10-01T19:41:44.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:44 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:44.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:44 smithi057 ceph-mon[114331]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:44.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:44 smithi057 ceph-mon[114331]: from='client.? 172.21.15.87:0/178474798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:41:44.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:44 smithi057 ceph-mon[114331]: osdmap e31: 6 total, 5 up, 6 in 2023-10-01T19:41:44.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:44 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:46.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:46 smithi087 ceph-mon[109386]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-10-01T19:41:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:46 smithi087 ceph-mon[109386]: osdmap e32: 6 total, 5 up, 6 in 2023-10-01T19:41:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:46 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:46.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:46 smithi057 ceph-mon[108286]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-10-01T19:41:46.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:46 smithi057 ceph-mon[108286]: osdmap e32: 6 total, 5 up, 6 in 2023-10-01T19:41:46.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:46 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:46.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:46 smithi057 ceph-mon[114331]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-10-01T19:41:46.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:46 smithi057 ceph-mon[114331]: osdmap e32: 6 total, 5 up, 6 in 2023-10-01T19:41:46.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:46 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:47.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:41:47 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:41:47] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:41:48.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:48 smithi087 ceph-mon[109386]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-10-01T19:41:48.603 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:48 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:48.603 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:48 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:48.603 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:48 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-01T19:41:48.603 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:48 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:48.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:48 smithi057 ceph-mon[114331]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-10-01T19:41:48.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:48 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:48.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:48 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:48.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:48 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-01T19:41:48.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:48 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:48.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:48 smithi057 ceph-mon[108286]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-10-01T19:41:48.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:48.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:48.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-01T19:41:48.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:48 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:49.533 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:49 smithi087 ceph-mon[109386]: Deploying daemon osd.5 on smithi087 2023-10-01T19:41:49.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:49 smithi057 ceph-mon[108286]: Deploying daemon osd.5 on smithi087 2023-10-01T19:41:49.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:49 smithi057 ceph-mon[114331]: Deploying daemon osd.5 on smithi087 2023-10-01T19:41:50.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:50 smithi057 ceph-mon[114331]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-10-01T19:41:50.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:50 smithi057 ceph-mon[108286]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-10-01T19:41:50.978 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:50 smithi087 ceph-mon[109386]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-10-01T19:41:52.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:52 smithi087 ceph-mon[109386]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-01T19:41:52.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:52 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:52.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:52 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:52.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:52 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:52.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:52 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:52.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:52 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:52.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:52 smithi057 ceph-mon[108286]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-01T19:41:52.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:52 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:52.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:52 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:52.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:52 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:52.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:52 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:52.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:52 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:52.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:52 smithi057 ceph-mon[114331]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-01T19:41:52.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:52 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:52.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:52 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:52.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:52 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:52.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:52 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:52.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:52 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:53.417 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 5 on host 'smithi087' 2023-10-01T19:41:53.789 DEBUG:teuthology.orchestra.run.smithi087:osd.5> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.5.service 2023-10-01T19:41:53.791 INFO:tasks.cephadm:Deploying osd.6 on smithi087 with /dev/vg_nvme/lv_2... 2023-10-01T19:41:53.791 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-01T19:41:53.821 INFO:journalctl@ceph.osd.5.smithi087.stdout:-- Logs begin at Sun 2023-10-01 19:27:12 UTC. -- 2023-10-01T19:41:54.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[114331]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-01T19:41:54.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:54.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:54.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:54.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:54.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:54.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:54.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[114331]: Adjusting osd_memory_target on smithi087 to 8044M 2023-10-01T19:41:54.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:54.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:54.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[108286]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-01T19:41:54.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[108286]: Adjusting osd_memory_target on smithi087 to 8044M 2023-10-01T19:41:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:54 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:54.744 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:54 smithi087 ceph-mon[109386]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-01T19:41:54.744 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:54 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:54.744 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:54 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:54.745 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:54 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:54.745 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:54 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:54.745 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:54 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:54.745 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:54 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:41:54.745 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:54 smithi087 ceph-mon[109386]: Adjusting osd_memory_target on smithi087 to 8044M 2023-10-01T19:41:54.745 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:54 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:54.745 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:54 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:55.133 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 19:41:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T19:41:54.742+0000 7f9fd67cf3c0 -1 osd.5 0 log_to_monitors true 2023-10-01T19:41:55.430 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-01T19:41:55.430 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-01T19:41:55.430 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2023-10-01T19:41:55.430 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2023-10-01T19:41:55.430 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121725 s, 861 MB/s 2023-10-01T19:41:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2023-10-01T19:41:55.687 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:41:55] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:41:55.869 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi087:vg_nvme/lv_2 2023-10-01T19:41:55.898 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:55 smithi087 ceph-mon[109386]: from='osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-01T19:41:55.898 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:55 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:55.898 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:55 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:55.898 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:55 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:55.899 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:55 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:55.899 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:55 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:55.899 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:55 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:56.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[108286]: from='osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-01T19:41:56.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:56.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:56.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:56.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:56.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:56.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:56.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[114331]: from='osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-01T19:41:56.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:56.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:56.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:56.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:56.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:56.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:55 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:57.099 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 19:41:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T19:41:56.705+0000 7f9fcd1d2700 -1 osd.5 0 waiting for initial osdmap 2023-10-01T19:41:57.099 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 19:41:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T19:41:56.710+0000 7f9fc6b67700 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-01T19:41:57.099 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:56 smithi087 ceph-mon[109386]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.099 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:56 smithi087 ceph-mon[109386]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.099 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:56 smithi087 ceph-mon[109386]: Detected new or changed devices on smithi087 2023-10-01T19:41:57.099 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:56 smithi087 ceph-mon[109386]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:56 smithi087 ceph-mon[109386]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:56 smithi087 ceph-mon[109386]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-10-01T19:41:57.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:56 smithi087 ceph-mon[109386]: from='osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-01T19:41:57.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:56 smithi087 ceph-mon[109386]: osdmap e33: 6 total, 5 up, 6 in 2023-10-01T19:41:57.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:56 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:57.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:56 smithi087 ceph-mon[109386]: from='osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:41:57.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:41:56] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:41:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[114331]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[114331]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[114331]: Detected new or changed devices on smithi087 2023-10-01T19:41:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[114331]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[114331]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[114331]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-10-01T19:41:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[114331]: from='osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-01T19:41:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[114331]: osdmap e33: 6 total, 5 up, 6 in 2023-10-01T19:41:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[114331]: from='osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:41:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[108286]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[108286]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[108286]: Detected new or changed devices on smithi087 2023-10-01T19:41:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[108286]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[108286]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[108286]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-10-01T19:41:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[108286]: from='osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-01T19:41:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[108286]: osdmap e33: 6 total, 5 up, 6 in 2023-10-01T19:41:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:56 smithi057 ceph-mon[108286]: from='osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:41:57.968 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-10-01T19:41:57.969 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: osdmap e34: 6 total, 5 up, 6 in 2023-10-01T19:41:57.969 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:57.969 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:57.969 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:57.969 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:57.969 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: Detected new or changed devices on smithi087 2023-10-01T19:41:57.969 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:57.969 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:57.969 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:57.969 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:57.969 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:57.970 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:41:57.970 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:41:57.970 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:41:57.970 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:57.970 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271] boot 2023-10-01T19:41:57.970 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: osdmap e35: 6 total, 6 up, 6 in 2023-10-01T19:41:57.970 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:57 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:58.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-10-01T19:41:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: osdmap e34: 6 total, 5 up, 6 in 2023-10-01T19:41:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: Detected new or changed devices on smithi087 2023-10-01T19:41:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:41:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:41:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:41:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271] boot 2023-10-01T19:41:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: osdmap e35: 6 total, 6 up, 6 in 2023-10-01T19:41:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:58.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-10-01T19:41:58.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: osdmap e34: 6 total, 5 up, 6 in 2023-10-01T19:41:58.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-10-01T19:41:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: Detected new or changed devices on smithi087 2023-10-01T19:41:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:41:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:41:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:41:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:41:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:41:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:41:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:41:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: osd.5 [v2:172.21.15.87:6808/3413643271,v1:172.21.15.87:6809/3413643271] boot 2023-10-01T19:41:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: osdmap e35: 6 total, 6 up, 6 in 2023-10-01T19:41:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:57 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:41:58.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:58 smithi087 ceph-mon[109386]: purged_snaps scrub starts 2023-10-01T19:41:58.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:58 smithi087 ceph-mon[109386]: purged_snaps scrub ok 2023-10-01T19:41:58.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:58 smithi087 ceph-mon[109386]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2023-10-01T19:41:58.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:58 smithi087 ceph-mon[109386]: Cluster is now healthy 2023-10-01T19:41:58.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:58 smithi087 ceph-mon[109386]: osdmap e36: 6 total, 6 up, 6 in 2023-10-01T19:41:59.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:58 smithi057 ceph-mon[108286]: purged_snaps scrub starts 2023-10-01T19:41:59.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:58 smithi057 ceph-mon[108286]: purged_snaps scrub ok 2023-10-01T19:41:59.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:58 smithi057 ceph-mon[108286]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2023-10-01T19:41:59.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:58 smithi057 ceph-mon[108286]: Cluster is now healthy 2023-10-01T19:41:59.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:58 smithi057 ceph-mon[108286]: osdmap e36: 6 total, 6 up, 6 in 2023-10-01T19:41:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:58 smithi057 ceph-mon[114331]: purged_snaps scrub starts 2023-10-01T19:41:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:58 smithi057 ceph-mon[114331]: purged_snaps scrub ok 2023-10-01T19:41:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:58 smithi057 ceph-mon[114331]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2023-10-01T19:41:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:58 smithi057 ceph-mon[114331]: Cluster is now healthy 2023-10-01T19:41:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:58 smithi057 ceph-mon[114331]: osdmap e36: 6 total, 6 up, 6 in 2023-10-01T19:42:00.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:59 smithi087 ceph-mon[109386]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:00.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:59 smithi087 ceph-mon[109386]: from='client.? 172.21.15.87:0/2157582837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2d554e8-b41d-4ddc-803d-6ad5de2c4b71"}]: dispatch 2023-10-01T19:42:00.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:59 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2d554e8-b41d-4ddc-803d-6ad5de2c4b71"}]: dispatch 2023-10-01T19:42:00.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:59 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2d554e8-b41d-4ddc-803d-6ad5de2c4b71"}]': finished 2023-10-01T19:42:00.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:59 smithi087 ceph-mon[109386]: osdmap e37: 7 total, 6 up, 7 in 2023-10-01T19:42:00.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:59 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:00.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:59 smithi087 ceph-mon[109386]: osdmap e38: 7 total, 6 up, 7 in 2023-10-01T19:42:00.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:41:59 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:00.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[108286]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:00.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[108286]: from='client.? 172.21.15.87:0/2157582837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2d554e8-b41d-4ddc-803d-6ad5de2c4b71"}]: dispatch 2023-10-01T19:42:00.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2d554e8-b41d-4ddc-803d-6ad5de2c4b71"}]: dispatch 2023-10-01T19:42:00.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2d554e8-b41d-4ddc-803d-6ad5de2c4b71"}]': finished 2023-10-01T19:42:00.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[108286]: osdmap e37: 7 total, 6 up, 7 in 2023-10-01T19:42:00.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:00.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[108286]: osdmap e38: 7 total, 6 up, 7 in 2023-10-01T19:42:00.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:00.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[114331]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:00.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[114331]: from='client.? 172.21.15.87:0/2157582837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2d554e8-b41d-4ddc-803d-6ad5de2c4b71"}]: dispatch 2023-10-01T19:42:00.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2d554e8-b41d-4ddc-803d-6ad5de2c4b71"}]: dispatch 2023-10-01T19:42:00.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2d554e8-b41d-4ddc-803d-6ad5de2c4b71"}]': finished 2023-10-01T19:42:00.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[114331]: osdmap e37: 7 total, 6 up, 7 in 2023-10-01T19:42:00.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:00.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[114331]: osdmap e38: 7 total, 6 up, 7 in 2023-10-01T19:42:00.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:41:59 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:01.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:00 smithi087 ceph-mon[109386]: from='client.? 172.21.15.87:0/1777882007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:42:01.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:00 smithi087 ceph-mon[109386]: osdmap e39: 7 total, 6 up, 7 in 2023-10-01T19:42:01.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:00 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:01.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:00 smithi057 ceph-mon[108286]: from='client.? 172.21.15.87:0/1777882007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:42:01.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:00 smithi057 ceph-mon[108286]: osdmap e39: 7 total, 6 up, 7 in 2023-10-01T19:42:01.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:00 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:01.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:00 smithi057 ceph-mon[114331]: from='client.? 172.21.15.87:0/1777882007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:42:01.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:00 smithi057 ceph-mon[114331]: osdmap e39: 7 total, 6 up, 7 in 2023-10-01T19:42:01.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:00 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:02.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:01 smithi087 ceph-mon[109386]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:02.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:01 smithi057 ceph-mon[108286]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:02.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:01 smithi057 ceph-mon[114331]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:04.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:03 smithi057 ceph-mon[108286]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:04.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:03 smithi057 ceph-mon[114331]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:04.221 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:03 smithi087 ceph-mon[109386]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:05.018 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-01T19:42:05.018 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:04 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:05.018 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:04 smithi087 ceph-mon[109386]: Deploying daemon osd.6 on smithi087 2023-10-01T19:42:05.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-01T19:42:05.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:04 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:05.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:04 smithi057 ceph-mon[108286]: Deploying daemon osd.6 on smithi087 2023-10-01T19:42:05.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-01T19:42:05.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:04 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:05.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:04 smithi057 ceph-mon[114331]: Deploying daemon osd.6 on smithi087 2023-10-01T19:42:06.981 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:06 smithi087 ceph-mon[109386]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:07.075 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:06 smithi057 ceph-mon[108286]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:07.075 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:06 smithi057 ceph-mon[114331]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:08.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:07 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:42:07] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:42:09.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:08 smithi087 ceph-mon[109386]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:09.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:09.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:09.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:09.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:09.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:09.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:09.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:08 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:09.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[108286]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:09.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:09.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:09.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:09.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:09.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:09.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:09.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:09.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[114331]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:09.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:09.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:09.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:09.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:09.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:09.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:09.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:08 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:09.598 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 6 on host 'smithi087' 2023-10-01T19:42:09.962 DEBUG:teuthology.orchestra.run.smithi087:osd.6> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.6.service 2023-10-01T19:42:10.005 INFO:tasks.cephadm:Deploying osd.7 on smithi087 with /dev/vg_nvme/lv_1... 2023-10-01T19:42:10.005 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-01T19:42:10.035 INFO:journalctl@ceph.osd.6.smithi087.stdout:-- Logs begin at Sun 2023-10-01 19:27:12 UTC. -- 2023-10-01T19:42:10.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:10 smithi087 ceph-mon[109386]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:10 smithi087 ceph-mon[109386]: Adjusting osd_memory_target on smithi087 to 5363M 2023-10-01T19:42:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:10 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:10.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[114331]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:10.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:10.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:10.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:10.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:10.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:10.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:10.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:10.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[114331]: Adjusting osd_memory_target on smithi087 to 5363M 2023-10-01T19:42:10.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:10.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[108286]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[108286]: Adjusting osd_memory_target on smithi087 to 5363M 2023-10-01T19:42:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:10 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:11.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 19:42:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T19:42:10.922+0000 7f413bc3c3c0 -1 osd.6 0 log_to_monitors true 2023-10-01T19:42:11.445 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-01T19:42:11.445 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-01T19:42:11.446 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2023-10-01T19:42:11.446 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2023-10-01T19:42:11.446 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125203 s, 837 MB/s 2023-10-01T19:42:11.446 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2023-10-01T19:42:11.774 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:11 smithi087 ceph-mon[109386]: from='osd.6 [v2:172.21.15.87:6816/1596710583,v1:172.21.15.87:6817/1596710583]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-01T19:42:11.774 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:11 smithi087 ceph-mon[109386]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-01T19:42:11.775 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi087:vg_nvme/lv_1 2023-10-01T19:42:11.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:11 smithi057 ceph-mon[108286]: from='osd.6 [v2:172.21.15.87:6816/1596710583,v1:172.21.15.87:6817/1596710583]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-01T19:42:11.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:11 smithi057 ceph-mon[108286]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-01T19:42:11.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:11 smithi057 ceph-mon[114331]: from='osd.6 [v2:172.21.15.87:6816/1596710583,v1:172.21.15.87:6817/1596710583]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-01T19:42:11.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:11 smithi057 ceph-mon[114331]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-01T19:42:12.743 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:12 smithi087 ceph-mon[109386]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:12.743 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:12 smithi087 ceph-mon[109386]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-01T19:42:12.744 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:12 smithi087 ceph-mon[109386]: osdmap e40: 7 total, 6 up, 7 in 2023-10-01T19:42:12.744 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:12 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:12.744 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:12 smithi087 ceph-mon[109386]: from='osd.6 [v2:172.21.15.87:6816/1596710583,v1:172.21.15.87:6817/1596710583]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:42:12.744 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:12 smithi087 ceph-mon[109386]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:42:12.744 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 19:42:12 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T19:42:12.635+0000 7f413263f700 -1 osd.6 0 waiting for initial osdmap 2023-10-01T19:42:12.744 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 19:42:12 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T19:42:12.643+0000 7f412cfd6700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-01T19:42:12.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:12 smithi057 ceph-mon[108286]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:12.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:12 smithi057 ceph-mon[108286]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-01T19:42:12.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:12 smithi057 ceph-mon[108286]: osdmap e40: 7 total, 6 up, 7 in 2023-10-01T19:42:12.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:12 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:12.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:12 smithi057 ceph-mon[108286]: from='osd.6 [v2:172.21.15.87:6816/1596710583,v1:172.21.15.87:6817/1596710583]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:42:12.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:12 smithi057 ceph-mon[108286]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:42:12.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:12 smithi057 ceph-mon[114331]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:12.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:12 smithi057 ceph-mon[114331]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-01T19:42:12.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:12 smithi057 ceph-mon[114331]: osdmap e40: 7 total, 6 up, 7 in 2023-10-01T19:42:12.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:12 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:12.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:12 smithi057 ceph-mon[114331]: from='osd.6 [v2:172.21.15.87:6816/1596710583,v1:172.21.15.87:6817/1596710583]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:42:12.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:12 smithi057 ceph-mon[114331]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:42:13.784 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:13 smithi087 ceph-mon[109386]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-10-01T19:42:13.784 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:13 smithi087 ceph-mon[109386]: osdmap e41: 7 total, 6 up, 7 in 2023-10-01T19:42:13.784 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:13.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:13.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:42:13.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:42:13.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:13.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:13 smithi087 ceph-mon[109386]: osd.6 [v2:172.21.15.87:6816/1596710583,v1:172.21.15.87:6817/1596710583] boot 2023-10-01T19:42:13.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:13 smithi087 ceph-mon[109386]: osdmap e42: 7 total, 7 up, 7 in 2023-10-01T19:42:13.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:13 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:13.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[108286]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-10-01T19:42:13.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[108286]: osdmap e41: 7 total, 6 up, 7 in 2023-10-01T19:42:13.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:13.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:13.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:42:13.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:42:13.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:13.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[108286]: osd.6 [v2:172.21.15.87:6816/1596710583,v1:172.21.15.87:6817/1596710583] boot 2023-10-01T19:42:13.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[108286]: osdmap e42: 7 total, 7 up, 7 in 2023-10-01T19:42:13.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:13.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[114331]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-10-01T19:42:13.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[114331]: osdmap e41: 7 total, 6 up, 7 in 2023-10-01T19:42:13.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:13.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:13.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-01T19:42:13.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-01T19:42:13.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:13.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[114331]: osd.6 [v2:172.21.15.87:6816/1596710583,v1:172.21.15.87:6817/1596710583] boot 2023-10-01T19:42:13.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[114331]: osdmap e42: 7 total, 7 up, 7 in 2023-10-01T19:42:13.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:13 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:15.129 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:14 smithi087 ceph-mon[109386]: purged_snaps scrub starts 2023-10-01T19:42:15.129 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:14 smithi087 ceph-mon[109386]: purged_snaps scrub ok 2023-10-01T19:42:15.130 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:14 smithi087 ceph-mon[109386]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:15.130 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:14 smithi087 ceph-mon[109386]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:15.130 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:14 smithi087 ceph-mon[109386]: osdmap e43: 7 total, 7 up, 7 in 2023-10-01T19:42:15.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:14 smithi057 ceph-mon[108286]: purged_snaps scrub starts 2023-10-01T19:42:15.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:14 smithi057 ceph-mon[108286]: purged_snaps scrub ok 2023-10-01T19:42:15.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:14 smithi057 ceph-mon[108286]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:15.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:14 smithi057 ceph-mon[108286]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:15.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:14 smithi057 ceph-mon[108286]: osdmap e43: 7 total, 7 up, 7 in 2023-10-01T19:42:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:14 smithi057 ceph-mon[114331]: purged_snaps scrub starts 2023-10-01T19:42:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:14 smithi057 ceph-mon[114331]: purged_snaps scrub ok 2023-10-01T19:42:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:14 smithi057 ceph-mon[114331]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-10-01T19:42:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:14 smithi057 ceph-mon[114331]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:15.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:14 smithi057 ceph-mon[114331]: osdmap e43: 7 total, 7 up, 7 in 2023-10-01T19:42:16.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:15 smithi087 ceph-mon[109386]: from='client.? 172.21.15.87:0/4115414755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccdaed0b-f228-491c-a150-5853328e7361"}]: dispatch 2023-10-01T19:42:16.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:15 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccdaed0b-f228-491c-a150-5853328e7361"}]: dispatch 2023-10-01T19:42:16.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:15 smithi087 ceph-mon[109386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccdaed0b-f228-491c-a150-5853328e7361"}]': finished 2023-10-01T19:42:16.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:15 smithi087 ceph-mon[109386]: osdmap e44: 8 total, 7 up, 8 in 2023-10-01T19:42:16.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:15 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:16.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:15 smithi087 ceph-mon[109386]: from='client.? 172.21.15.87:0/2211129078' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:42:16.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:15 smithi057 ceph-mon[108286]: from='client.? 172.21.15.87:0/4115414755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccdaed0b-f228-491c-a150-5853328e7361"}]: dispatch 2023-10-01T19:42:16.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:15 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccdaed0b-f228-491c-a150-5853328e7361"}]: dispatch 2023-10-01T19:42:16.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:15 smithi057 ceph-mon[108286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccdaed0b-f228-491c-a150-5853328e7361"}]': finished 2023-10-01T19:42:16.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:15 smithi057 ceph-mon[108286]: osdmap e44: 8 total, 7 up, 8 in 2023-10-01T19:42:16.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:15 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:16.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:15 smithi057 ceph-mon[108286]: from='client.? 172.21.15.87:0/2211129078' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:42:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:15 smithi057 ceph-mon[114331]: from='client.? 172.21.15.87:0/4115414755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccdaed0b-f228-491c-a150-5853328e7361"}]: dispatch 2023-10-01T19:42:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:15 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccdaed0b-f228-491c-a150-5853328e7361"}]: dispatch 2023-10-01T19:42:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:15 smithi057 ceph-mon[114331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccdaed0b-f228-491c-a150-5853328e7361"}]': finished 2023-10-01T19:42:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:15 smithi057 ceph-mon[114331]: osdmap e44: 8 total, 7 up, 8 in 2023-10-01T19:42:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:15 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:15 smithi057 ceph-mon[114331]: from='client.? 172.21.15.87:0/2211129078' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-01T19:42:17.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:16 smithi087 ceph-mon[109386]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-10-01T19:42:17.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:16 smithi087 ceph-mon[109386]: osdmap e45: 8 total, 7 up, 8 in 2023-10-01T19:42:17.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:16 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:17.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:42:16] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:42:17.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:16 smithi057 ceph-mon[108286]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-10-01T19:42:17.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:16 smithi057 ceph-mon[108286]: osdmap e45: 8 total, 7 up, 8 in 2023-10-01T19:42:17.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:16 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:17.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:16 smithi057 ceph-mon[114331]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-10-01T19:42:17.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:16 smithi057 ceph-mon[114331]: osdmap e45: 8 total, 7 up, 8 in 2023-10-01T19:42:17.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:16 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:18.119 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:17 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:18.120 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:17 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:18.120 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:17 smithi087 ceph-mon[109386]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-10-01T19:42:18.120 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:17 smithi087 ceph-mon[109386]: osdmap e46: 8 total, 7 up, 8 in 2023-10-01T19:42:18.120 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:17 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:17 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:17 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:17 smithi057 ceph-mon[108286]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-10-01T19:42:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:17 smithi057 ceph-mon[108286]: osdmap e46: 8 total, 7 up, 8 in 2023-10-01T19:42:18.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:17 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:18.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:17 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:18.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:17 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:18.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:17 smithi057 ceph-mon[114331]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-10-01T19:42:18.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:17 smithi057 ceph-mon[114331]: osdmap e46: 8 total, 7 up, 8 in 2023-10-01T19:42:18.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:17 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:20.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:20 smithi087 ceph-mon[109386]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-10-01T19:42:20.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:20 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-01T19:42:20.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:20 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:21.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:20 smithi057 ceph-mon[108286]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-10-01T19:42:21.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:20 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-01T19:42:21.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:20 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:21.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:20 smithi057 ceph-mon[114331]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-10-01T19:42:21.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:20 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-01T19:42:21.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:20 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:21 smithi087 ceph-mon[109386]: Deploying daemon osd.7 on smithi087 2023-10-01T19:42:22.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:21 smithi057 ceph-mon[108286]: Deploying daemon osd.7 on smithi087 2023-10-01T19:42:22.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:21 smithi057 ceph-mon[114331]: Deploying daemon osd.7 on smithi087 2023-10-01T19:42:22.828 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:22 smithi087 ceph-mon[109386]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-10-01T19:42:23.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:22 smithi057 ceph-mon[108286]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-10-01T19:42:23.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:22 smithi057 ceph-mon[114331]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-10-01T19:42:24.841 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:24 smithi087 ceph-mon[109386]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-01T19:42:24.841 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:24 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:24.841 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:24 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:24.842 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:24 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:24.842 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:24 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:24.842 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:24 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:25.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:24 smithi057 ceph-mon[114331]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-01T19:42:25.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:24 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:25.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:24 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:25.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:24 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:25.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:24 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:25.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:24 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:25.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:24 smithi057 ceph-mon[108286]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-01T19:42:25.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:24 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:25.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:24 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:25.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:24 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:25.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:24 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:25.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:24 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:26.003 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 7 on host 'smithi087' 2023-10-01T19:42:26.369 DEBUG:teuthology.orchestra.run.smithi087:osd.7> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.7.service 2023-10-01T19:42:26.371 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2023-10-01T19:42:26.371 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-01T19:42:26.402 INFO:journalctl@ceph.osd.7.smithi087.stdout:-- Logs begin at Sun 2023-10-01 19:27:12 UTC. -- 2023-10-01T19:42:27.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[114331]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-01T19:42:27.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:27.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:27.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:27.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:27.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:27.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:27.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:27.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:27.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:27.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:27.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[108286]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-01T19:42:27.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:27.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:27.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:27.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:27.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:27.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:27.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:27.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:27.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:27.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:26 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:27.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:26 smithi087 ceph-mon[109386]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-01T19:42:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:27.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:27.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:26 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:27.632 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 19:42:27 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T19:42:27.212+0000 7fe0dae9c3c0 -1 osd.7 0 log_to_monitors true 2023-10-01T19:42:28.023 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:27 smithi057 ceph-mon[108286]: Adjusting osd_memory_target on smithi087 to 4022M 2023-10-01T19:42:28.023 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:27 smithi057 ceph-mon[108286]: from='osd.7 [v2:172.21.15.87:6824/1585249784,v1:172.21.15.87:6825/1585249784]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-01T19:42:28.023 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:27 smithi057 ceph-mon[108286]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-01T19:42:28.023 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:27 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:42:28.023 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:27 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:42:28.024 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:27 smithi057 ceph-mon[114331]: Adjusting osd_memory_target on smithi087 to 4022M 2023-10-01T19:42:28.024 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:27 smithi057 ceph-mon[114331]: from='osd.7 [v2:172.21.15.87:6824/1585249784,v1:172.21.15.87:6825/1585249784]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-01T19:42:28.024 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:27 smithi057 ceph-mon[114331]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-01T19:42:28.024 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:27 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:42:28.024 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:27 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:42:28.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:27 smithi087 ceph-mon[109386]: Adjusting osd_memory_target on smithi087 to 4022M 2023-10-01T19:42:28.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:27 smithi087 ceph-mon[109386]: from='osd.7 [v2:172.21.15.87:6824/1585249784,v1:172.21.15.87:6825/1585249784]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-01T19:42:28.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:27 smithi087 ceph-mon[109386]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-01T19:42:28.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:27 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:42:28.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:27 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:42:28.198 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:42:28.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:42:28] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:42:28.671 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":47,"num_osds":8,"num_up_osds":7,"osd_up_since":1696189333,"num_in_osds":8,"osd_in_since":1696189334,"num_remapped_pgs":0} 2023-10-01T19:42:29.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 19:42:28 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T19:42:28.716+0000 7fe0d189f700 -1 osd.7 0 waiting for initial osdmap 2023-10-01T19:42:29.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 19:42:28 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T19:42:28.726+0000 7fe0ca232700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-01T19:42:29.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:28 smithi087 ceph-mon[109386]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-10-01T19:42:29.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:28 smithi087 ceph-mon[109386]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-01T19:42:29.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:28 smithi087 ceph-mon[109386]: osdmap e47: 8 total, 7 up, 8 in 2023-10-01T19:42:29.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:28 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:29.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:28 smithi087 ceph-mon[109386]: from='osd.7 [v2:172.21.15.87:6824/1585249784,v1:172.21.15.87:6825/1585249784]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:42:29.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:28 smithi087 ceph-mon[109386]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:42:29.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:28 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:29.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:28 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:29.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:28 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/4283300245' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-01T19:42:29.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[114331]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-10-01T19:42:29.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[114331]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-01T19:42:29.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[114331]: osdmap e47: 8 total, 7 up, 8 in 2023-10-01T19:42:29.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:29.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[114331]: from='osd.7 [v2:172.21.15.87:6824/1585249784,v1:172.21.15.87:6825/1585249784]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:42:29.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[114331]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:42:29.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:29.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:29.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/4283300245' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-01T19:42:29.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[108286]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-10-01T19:42:29.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[108286]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-01T19:42:29.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[108286]: osdmap e47: 8 total, 7 up, 8 in 2023-10-01T19:42:29.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:29.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[108286]: from='osd.7 [v2:172.21.15.87:6824/1585249784,v1:172.21.15.87:6825/1585249784]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:42:29.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[108286]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-10-01T19:42:29.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:29.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:29.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:28 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/4283300245' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-01T19:42:29.672 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-01T19:42:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[108286]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-10-01T19:42:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[108286]: osdmap e48: 8 total, 7 up, 8 in 2023-10-01T19:42:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[108286]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-10-01T19:42:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[108286]: osd.7 [v2:172.21.15.87:6824/1585249784,v1:172.21.15.87:6825/1585249784] boot 2023-10-01T19:42:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[108286]: osdmap e49: 8 total, 8 up, 8 in 2023-10-01T19:42:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:29.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[114331]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-10-01T19:42:29.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[114331]: osdmap e48: 8 total, 7 up, 8 in 2023-10-01T19:42:29.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:29.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:29.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[114331]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-10-01T19:42:29.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[114331]: osd.7 [v2:172.21.15.87:6824/1585249784,v1:172.21.15.87:6825/1585249784] boot 2023-10-01T19:42:29.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[114331]: osdmap e49: 8 total, 8 up, 8 in 2023-10-01T19:42:29.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:29 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:30.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:29 smithi087 ceph-mon[109386]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-10-01T19:42:30.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:29 smithi087 ceph-mon[109386]: osdmap e48: 8 total, 7 up, 8 in 2023-10-01T19:42:30.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:29 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:30.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:29 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:30.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:29 smithi087 ceph-mon[109386]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-10-01T19:42:30.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:29 smithi087 ceph-mon[109386]: osd.7 [v2:172.21.15.87:6824/1585249784,v1:172.21.15.87:6825/1585249784] boot 2023-10-01T19:42:30.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:29 smithi087 ceph-mon[109386]: osdmap e49: 8 total, 8 up, 8 in 2023-10-01T19:42:30.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:29 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:31.083 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:42:31.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:30 smithi057 ceph-mon[108286]: purged_snaps scrub starts 2023-10-01T19:42:31.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:30 smithi057 ceph-mon[108286]: purged_snaps scrub ok 2023-10-01T19:42:31.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:30 smithi057 ceph-mon[108286]: osdmap e50: 8 total, 8 up, 8 in 2023-10-01T19:42:31.107 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:30 smithi057 ceph-mon[114331]: purged_snaps scrub starts 2023-10-01T19:42:31.107 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:30 smithi057 ceph-mon[114331]: purged_snaps scrub ok 2023-10-01T19:42:31.107 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:30 smithi057 ceph-mon[114331]: osdmap e50: 8 total, 8 up, 8 in 2023-10-01T19:42:31.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:30 smithi087 ceph-mon[109386]: purged_snaps scrub starts 2023-10-01T19:42:31.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:30 smithi087 ceph-mon[109386]: purged_snaps scrub ok 2023-10-01T19:42:31.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:30 smithi087 ceph-mon[109386]: osdmap e50: 8 total, 8 up, 8 in 2023-10-01T19:42:31.463 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":50,"num_osds":8,"num_up_osds":8,"osd_up_since":1696189349,"num_in_osds":8,"osd_in_since":1696189334,"num_remapped_pgs":1} 2023-10-01T19:42:31.464 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-01T19:42:31.724 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:42:31.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:31 smithi057 ceph-mon[114331]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:31.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:31 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1346123754' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-01T19:42:31.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:31 smithi057 ceph-mon[114331]: osdmap e51: 8 total, 8 up, 8 in 2023-10-01T19:42:31.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:31 smithi057 ceph-mon[108286]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:31.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:31 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1346123754' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-01T19:42:31.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:31 smithi057 ceph-mon[108286]: osdmap e51: 8 total, 8 up, 8 in 2023-10-01T19:42:32.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:31 smithi087 ceph-mon[109386]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:31 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1346123754' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-01T19:42:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:31 smithi087 ceph-mon[109386]: osdmap e51: 8 total, 8 up, 8 in 2023-10-01T19:42:32.924 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:42:32.924 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":52,"fsid":"0f6d5dc0-6092-11ee-8db5-212e2dc638e7","created":"2023-10-01T19:38:40.968670+0000","modified":"2023-10-01T19:42:32.734768+0000","last_up_change":"2023-10-01T19:42:29.672336+0000","last_in_change":"2023-10-01T19:42:14.852777+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-01T19:41:08.148780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"8c7f9d93-03f1-48e1-9ac3-eab5bbd20afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3459964208},{"type":"v1","addr":"172.21.15.57:6803","nonce":3459964208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3459964208},{"type":"v1","addr":"172.21.15.57:6805","nonce":3459964208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3459964208},{"type":"v1","addr":"172.21.15.57:6809","nonce":3459964208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3459964208},{"type":"v1","addr":"172.21.15.57:6807","nonce":3459964208}]},"public_addr":"172.21.15.57:6803/3459964208","cluster_addr":"172.21.15.57:6805/3459964208","heartbeat_back_addr":"172.21.15.57:6809/3459964208","heartbeat_front_addr":"172.21.15.57:6807/3459964208","state":["exists","up"]},{"osd":1,"uuid":"2272344b-6026-4653-ab79-e9c746993364","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":3617577730},{"type":"v1","addr":"172.21.15.57:6811","nonce":3617577730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":3617577730},{"type":"v1","addr":"172.21.15.57:6813","nonce":3617577730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":3617577730},{"type":"v1","addr":"172.21.15.57:6817","nonce":3617577730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":3617577730},{"type":"v1","addr":"172.21.15.57:6815","nonce":3617577730}]},"public_addr":"172.21.15.57:6811/3617577730","cluster_addr":"172.21.15.57:6813/3617577730","heartbeat_back_addr":"172.21.15.57:6817/3617577730","heartbeat_front_addr":"172.21.15.57:6815/3617577730","state":["exists","up"]},{"osd":2,"uuid":"63bc952e-d249-4ae3-a4cb-a50f415d9beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2187972944},{"type":"v1","addr":"172.21.15.57:6819","nonce":2187972944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2187972944},{"type":"v1","addr":"172.21.15.57:6821","nonce":2187972944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2187972944},{"type":"v1","addr":"172.21.15.57:6825","nonce":2187972944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2187972944},{"type":"v1","addr":"172.21.15.57:6823","nonce":2187972944}]},"public_addr":"172.21.15.57:6819/2187972944","cluster_addr":"172.21.15.57:6821/2187972944","heartbeat_back_addr":"172.21.15.57:6825/2187972944","heartbeat_front_addr":"172.21.15.57:6823/2187972944","state":["exists","up"]},{"osd":3,"uuid":"039ec24f-b99e-4248-b0d9-b9e61bfd8833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1040473437},{"type":"v1","addr":"172.21.15.57:6827","nonce":1040473437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1040473437},{"type":"v1","addr":"172.21.15.57:6829","nonce":1040473437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1040473437},{"type":"v1","addr":"172.21.15.57:6833","nonce":1040473437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1040473437},{"type":"v1","addr":"172.21.15.57:6831","nonce":1040473437}]},"public_addr":"172.21.15.57:6827/1040473437","cluster_addr":"172.21.15.57:6829/1040473437","heartbeat_back_addr":"172.21.15.57:6833/1040473437","heartbeat_front_addr":"172.21.15.57:6831/1040473437","state":["exists","up"]},{"osd":4,"uuid":"709bab5f-07ff-4bf9-9c9c-942bbc4901a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2800415383},{"type":"v1","addr":"172.21.15.87:6801","nonce":2800415383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":2800415383},{"type":"v1","addr":"172.21.15.87:6803","nonce":2800415383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":2800415383},{"type":"v1","addr":"172.21.15.87:6807","nonce":2800415383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":2800415383},{"type":"v1","addr":"172.21.15.87:6805","nonce":2800415383}]},"public_addr":"172.21.15.87:6801/2800415383","cluster_addr":"172.21.15.87:6803/2800415383","heartbeat_back_addr":"172.21.15.87:6807/2800415383","heartbeat_front_addr":"172.21.15.87:6805/2800415383","state":["exists","up"]},{"osd":5,"uuid":"839f46cd-0e7c-4f51-b784-3c85cc3dde2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3413643271},{"type":"v1","addr":"172.21.15.87:6809","nonce":3413643271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3413643271},{"type":"v1","addr":"172.21.15.87:6811","nonce":3413643271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3413643271},{"type":"v1","addr":"172.21.15.87:6815","nonce":3413643271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3413643271},{"type":"v1","addr":"172.21.15.87:6813","nonce":3413643271}]},"public_addr":"172.21.15.87:6809/3413643271","cluster_addr":"172.21.15.87:6811/3413643271","heartbeat_back_addr":"172.21.15.87:6815/3413643271","heartbeat_front_addr":"172.21.15.87:6813/3413643271","state":["exists","up"]},{"osd":6,"uuid":"d2d554e8-b41d-4ddc-803d-6ad5de2c4b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1596710583},{"type":"v1","addr":"172.21.15.87:6817","nonce":1596710583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1596710583},{"type":"v1","addr":"172.21.15.87:6819","nonce":1596710583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1596710583},{"type":"v1","addr":"172.21.15.87:6823","nonce":1596710583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1596710583},{"type":"v1","addr":"172.21.15.87:6821","nonce":1596710583}]},"public_addr":"172.21.15.87:6817/1596710583","cluster_addr":"172.21.15.87:6819/1596710583","heartbeat_back_addr":"172.21.15.87:6823/1596710583","heartbeat_front_addr":"172.21.15.87:6821/1596710583","state":["exists","up"]},{"osd":7,"uuid":"ccdaed0b-f228-491c-a150-5853328e7361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1585249784},{"type":"v1","addr":"172.21.15.87:6825","nonce":1585249784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1585249784},{"type":"v1","addr":"172.21.15.87:6827","nonce":1585249784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1585249784},{"type":"v1","addr":"172.21.15.87:6831","nonce":1585249784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1585249784},{"type":"v1","addr":"172.21.15.87:6829","nonce":1585249784}]},"public_addr":"172.21.15.87:6825/1585249784","cluster_addr":"172.21.15.87:6827/1585249784","heartbeat_back_addr":"172.21.15.87:6831/1585249784","heartbeat_front_addr":"172.21.15.87:6829/1585249784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:40:29.458860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:40:47.602237+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:41:05.611067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:41:23.644060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:41:39.539090+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:41:55.722558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:42:11.880707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:42:28.220193+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/427084745":"2023-10-02T19:39:27.540639+0000","172.21.15.57:0/589184120":"2023-10-02T19:39:27.540639+0000","172.21.15.57:0/19155294":"2023-10-02T19:39:27.540639+0000","172.21.15.57:6800/3618988359":"2023-10-02T19:39:27.540639+0000","172.21.15.57:0/1453123188":"2023-10-02T19:39:05.183540+0000","172.21.15.57:0/34608214":"2023-10-02T19:39:05.183540+0000","172.21.15.57:6801/3618988359":"2023-10-02T19:39:27.540639+0000","172.21.15.57:6800/1407753143":"2023-10-02T19:39:05.183540+0000","172.21.15.57:0/69774414":"2023-10-02T19:39:05.183540+0000","172.21.15.57:6801/1407753143":"2023-10-02T19:39:05.183540+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-01T19:42:33.322 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-01T19:41:08.148780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-01T19:42:33.323 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-01T19:42:33.575 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:42:34.050 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:33 smithi057 ceph-mon[108286]: osdmap e52: 8 total, 8 up, 8 in 2023-10-01T19:42:34.051 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:33 smithi057 ceph-mon[108286]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:34.051 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:33 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3456162741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-01T19:42:34.051 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:33 smithi057 ceph-mon[114331]: osdmap e52: 8 total, 8 up, 8 in 2023-10-01T19:42:34.051 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:33 smithi057 ceph-mon[114331]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:34.051 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:33 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3456162741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-01T19:42:34.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:33 smithi087 ceph-mon[109386]: osdmap e52: 8 total, 8 up, 8 in 2023-10-01T19:42:34.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:33 smithi087 ceph-mon[109386]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:34.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:33 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3456162741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-01T19:42:34.821 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-01T19:42:35.129 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:34 smithi057 ceph-mon[108286]: osdmap e53: 8 total, 8 up, 8 in 2023-10-01T19:42:35.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:34 smithi057 ceph-mon[114331]: osdmap e53: 8 total, 8 up, 8 in 2023-10-01T19:42:35.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:34 smithi087 ceph-mon[109386]: osdmap e53: 8 total, 8 up, 8 in 2023-10-01T19:42:35.201 INFO:tasks.cephadm:Adding prometheus.a on smithi087 2023-10-01T19:42:35.202 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch apply prometheus '1;smithi087=a' 2023-10-01T19:42:35.773 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:35 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2440463771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-01T19:42:35.773 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:35 smithi087 ceph-mon[109386]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-10-01T19:42:36.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:35 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2440463771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-01T19:42:36.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:35 smithi057 ceph-mon[108286]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-10-01T19:42:36.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:35 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2440463771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-01T19:42:36.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:35 smithi057 ceph-mon[114331]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-10-01T19:42:36.526 INFO:teuthology.orchestra.run.smithi087.stdout:Scheduled prometheus update... 2023-10-01T19:42:37.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:36 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:42:36] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:42:37.251 DEBUG:teuthology.orchestra.run.smithi087:prometheus.a> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@prometheus.a.service 2023-10-01T19:42:37.253 INFO:tasks.cephadm:Adding node-exporter.a on smithi057 2023-10-01T19:42:37.254 INFO:tasks.cephadm:Adding node-exporter.b on smithi087 2023-10-01T19:42:37.254 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch apply node-exporter '2;smithi057=a;smithi087=b' 2023-10-01T19:42:37.287 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:-- Logs begin at Sun 2023-10-01 19:27:12 UTC. -- 2023-10-01T19:42:37.821 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-mon[109386]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi087=a", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:37.821 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-mon[109386]: Saving service prometheus spec with placement smithi087=a;count:1 2023-10-01T19:42:37.821 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:37.821 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:37.821 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:37.822 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:37.822 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:37.822 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2023-10-01T19:42:37.822 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:37.822 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:37.822 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ignoring --setuser ceph since I am not root 2023-10-01T19:42:37.822 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ignoring --setgroup ceph since I am not root 2023-10-01T19:42:37.822 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-mgr[111192]: -- 172.21.15.87:0/3130181015 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556dcac18340 con 0x556dcabf0c00 2023-10-01T19:42:37.823 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:37.728+0000 7f50eba82000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T19:42:37.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ignoring --setuser ceph since I am not root 2023-10-01T19:42:37.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ignoring --setgroup ceph since I am not root 2023-10-01T19:42:37.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mgr[108556]: -- 172.21.15.57:0/1507967127 <== mon.1 v2:172.21.15.57:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b66118340 con 0x561b660f0800 2023-10-01T19:42:37.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:37.736+0000 7f9d47af7000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T19:42:37.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[114331]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi087=a", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:37.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[114331]: Saving service prometheus spec with placement smithi087=a;count:1 2023-10-01T19:42:37.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:37.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:37.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:37.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:37.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:37.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2023-10-01T19:42:37.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:37.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[108286]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi087=a", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[108286]: Saving service prometheus spec with placement smithi087=a;count:1 2023-10-01T19:42:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2023-10-01T19:42:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' 2023-10-01T19:42:38.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:37.899+0000 7f50eba82000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T19:42:38.261 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:37 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:37.927+0000 7f9d47af7000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T19:42:38.484 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:38 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:38.221+0000 7f50eba82000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T19:42:38.554 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:38 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:38.261+0000 7f9d47af7000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T19:42:38.742 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:38 smithi087 ceph-mon[109386]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2023-10-01T19:42:38.742 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:38 smithi087 ceph-mon[109386]: mgrmap e16: y(active, since 3m), standbys: x 2023-10-01T19:42:38.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:38 smithi057 ceph-mon[108286]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2023-10-01T19:42:38.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:38 smithi057 ceph-mon[108286]: mgrmap e16: y(active, since 3m), standbys: x 2023-10-01T19:42:38.829 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:38 smithi057 ceph-mon[114331]: from='mgr.14152 172.21.15.57:0/3705600998' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2023-10-01T19:42:38.829 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:38 smithi057 ceph-mon[114331]: mgrmap e16: y(active, since 3m), standbys: x 2023-10-01T19:42:39.097 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:38 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:38.741+0000 7f50eba82000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T19:42:39.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:38 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:38.828+0000 7f9d47af7000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T19:42:39.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:39 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:39.236+0000 7f9d47af7000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T19:42:40.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:39 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:39.097+0000 7f50eba82000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T19:42:40.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:39 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:39.734+0000 7f50eba82000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T19:42:40.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:39 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:39.899+0000 7f50eba82000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T19:42:40.422 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:39 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:39.971+0000 7f9d47af7000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T19:42:40.422 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:40 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:40.164+0000 7f9d47af7000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T19:42:40.748 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:40.489+0000 7f50eba82000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T19:42:40.749 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:40.571+0000 7f50eba82000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T19:42:40.749 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:40.657+0000 7f50eba82000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T19:42:40.749 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:40.748+0000 7f50eba82000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T19:42:41.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:40.837+0000 7f50eba82000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T19:42:41.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:40.927+0000 7f50eba82000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T19:42:41.153 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:40 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:40.857+0000 7f9d47af7000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T19:42:41.153 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:40 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:40.951+0000 7f9d47af7000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T19:42:41.154 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:41.048+0000 7f9d47af7000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T19:42:41.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:41.152+0000 7f9d47af7000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T19:42:41.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:41.255+0000 7f9d47af7000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T19:42:41.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:41.359+0000 7f9d47af7000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T19:42:41.495 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:41 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:41.222+0000 7f50eba82000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T19:42:41.495 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:41 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:41.315+0000 7f50eba82000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T19:42:41.496 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:41 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:41.495+0000 7f50eba82000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T19:42:41.882 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:41 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:41.579+0000 7f50eba82000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T19:42:42.015 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:41.703+0000 7f9d47af7000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T19:42:42.015 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:41.809+0000 7f9d47af7000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T19:42:42.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:42 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:42.014+0000 7f9d47af7000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T19:42:42.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:42 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:42.110+0000 7f9d47af7000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T19:42:42.550 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:42 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:42.271+0000 7f50eba82000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T19:42:42.550 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:42 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:42.417+0000 7f50eba82000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T19:42:42.550 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:42 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:42.550+0000 7f50eba82000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:42:43.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:42 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:42.914+0000 7f9d47af7000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T19:42:43.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:43.080+0000 7f9d47af7000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T19:42:43.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:42 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:42.939+0000 7f50eba82000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T19:42:43.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:43.073+0000 7f50eba82000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:42:43.638 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:43.232+0000 7f9d47af7000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:42:43.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-mon[109386]: Standby manager daemon x restarted 2023-10-01T19:42:43.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-mon[109386]: Standby manager daemon x started 2023-10-01T19:42:43.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/2927595979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T19:42:43.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/2927595979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:42:43.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/2927595979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T19:42:43.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/2927595979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:42:43.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:43.452+0000 7f50eba82000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T19:42:43.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:43.566+0000 7f50eba82000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T19:42:43.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: [01/Oct/2023:19:42:43] ENGINE Bus STARTING 2023-10-01T19:42:43.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: CherryPy Checker: 2023-10-01T19:42:43.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: The Application mounted at '' has an empty config. 2023-10-01T19:42:43.884 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: 2023-10-01T19:42:43.884 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: [01/Oct/2023:19:42:43] ENGINE Serving on http://:::9283 2023-10-01T19:42:43.884 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:42:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: [01/Oct/2023:19:42:43] ENGINE Bus STARTED 2023-10-01T19:42:43.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:43.688+0000 7f9d47af7000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T19:42:43.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:43.844+0000 7f9d47af7000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:42:43.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-mon[108286]: Standby manager daemon x restarted 2023-10-01T19:42:43.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-mon[108286]: Standby manager daemon x started 2023-10-01T19:42:43.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/2927595979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T19:42:43.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/2927595979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:42:43.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/2927595979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T19:42:43.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/2927595979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:42:43.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-mon[114331]: Standby manager daemon x restarted 2023-10-01T19:42:43.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-mon[114331]: Standby manager daemon x started 2023-10-01T19:42:43.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/2927595979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T19:42:43.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/2927595979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:42:43.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/2927595979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T19:42:43.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:43 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/2927595979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:42:44.653 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:44.287+0000 7f9d47af7000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T19:42:44.653 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:44.417+0000 7f9d47af7000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T19:42:44.919 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:44 smithi087 ceph-mon[109386]: mgrmap e17: y(active, since 3m), standbys: x 2023-10-01T19:42:44.919 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:44 smithi087 ceph-mon[109386]: Active manager daemon y restarted 2023-10-01T19:42:44.919 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:44 smithi087 ceph-mon[109386]: Activating manager daemon y 2023-10-01T19:42:44.919 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:44 smithi087 ceph-mon[109386]: osdmap e54: 8 total, 8 up, 8 in 2023-10-01T19:42:44.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-mon[108286]: mgrmap e17: y(active, since 3m), standbys: x 2023-10-01T19:42:44.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-mon[108286]: Active manager daemon y restarted 2023-10-01T19:42:44.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-mon[108286]: Activating manager daemon y 2023-10-01T19:42:44.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-mon[108286]: osdmap e54: 8 total, 8 up, 8 in 2023-10-01T19:42:44.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-mon[114331]: mgrmap e17: y(active, since 3m), standbys: x 2023-10-01T19:42:44.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-mon[114331]: Active manager daemon y restarted 2023-10-01T19:42:44.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-mon[114331]: Activating manager daemon y 2023-10-01T19:42:44.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-mon[114331]: osdmap e54: 8 total, 8 up, 8 in 2023-10-01T19:42:44.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:42:44] ENGINE Bus STARTING 2023-10-01T19:42:44.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: CherryPy Checker: 2023-10-01T19:42:44.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: The Application mounted at '' has an empty config. 2023-10-01T19:42:44.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:42:44.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:42:44] ENGINE Serving on http://:::9283 2023-10-01T19:42:44.923 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:42:44] ENGINE Bus STARTED 2023-10-01T19:42:45.702 INFO:teuthology.orchestra.run.smithi087.stdout:Scheduled node-exporter update... 2023-10-01T19:42:45.910 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: mgrmap e18: y(active, starting, since 0.216027s), standbys: x 2023-10-01T19:42:45.910 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:42:45.910 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:42:45.910 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:42:45.911 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:42:45.911 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T19:42:45.911 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:42:45.911 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:42:45.911 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:42:45.911 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:42:45.911 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:42:45.911 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:42:45.911 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:45.911 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:45.911 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:42:45.911 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:42:45.912 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:42:45.912 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: Manager daemon y is now available 2023-10-01T19:42:45.912 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.912 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:45.912 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:45.912 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:45.912 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:45.912 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:45.912 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:45.912 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]': finished 2023-10-01T19:42:45.912 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:42:45.912 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]': finished 2023-10-01T19:42:45.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:42:45.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:42:45.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:42:45.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:42:45.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:42:45.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2023-10-01T19:42:45.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: mgrmap e18: y(active, starting, since 0.216027s), standbys: x 2023-10-01T19:42:45.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:42:45.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:42:45.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:42:45.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:42:45.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T19:42:45.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:42:45.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:42:45.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:42:45.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:42:45.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:42:45.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:42:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:42:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:42:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:42:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: Manager daemon y is now available 2023-10-01T19:42:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:45.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:45.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]': finished 2023-10-01T19:42:45.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:42:45.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: mgrmap e18: y(active, starting, since 0.216027s), standbys: x 2023-10-01T19:42:45.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:42:45.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:42:45.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:42:45.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:42:45.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T19:42:45.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:42:45.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:42:45.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:42:45.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:42:45.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:42:45.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:42:45.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:42:45.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:42:45.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:42:45.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:42:45.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:42:45.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: Manager daemon y is now available 2023-10-01T19:42:45.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:45.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:45.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:45.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:45.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:45.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:42:45.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]': finished 2023-10-01T19:42:45.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:42:45.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]': finished 2023-10-01T19:42:45.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:42:45.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:42:45.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:42:45.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:42:45.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:42:45.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2023-10-01T19:42:45.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]': finished 2023-10-01T19:42:45.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:42:45.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:42:45.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:42:45.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:42:45.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:42:45.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:45.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2023-10-01T19:42:46.073 DEBUG:teuthology.orchestra.run.smithi057:node-exporter.a> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.a.service 2023-10-01T19:42:46.076 DEBUG:teuthology.orchestra.run.smithi087:node-exporter.b> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.b.service 2023-10-01T19:42:46.078 INFO:tasks.cephadm:Adding alertmanager.a on smithi057 2023-10-01T19:42:46.078 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch apply alertmanager '1;smithi057=a' 2023-10-01T19:42:46.106 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:-- Logs begin at Sun 2023-10-01 19:27:22 UTC. -- 2023-10-01T19:42:46.107 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:-- Logs begin at Sun 2023-10-01 19:27:12 UTC. -- 2023-10-01T19:42:46.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:42:46] ENGINE Bus STARTING 2023-10-01T19:42:46.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:42:46] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:42:46.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:42:46] ENGINE Bus STARTED 2023-10-01T19:42:46.976 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:46 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2023-10-01T19:42:46.977 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:46 smithi087 ceph-mon[109386]: mgrmap e19: y(active, since 1.25434s), standbys: x 2023-10-01T19:42:46.977 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:46 smithi087 ceph-mon[109386]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi057=a;smithi087=b", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:46.977 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:46 smithi087 ceph-mon[109386]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:46.977 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:46 smithi087 ceph-mon[109386]: Saving service node-exporter spec with placement smithi057=a;smithi087=b;count:2 2023-10-01T19:42:46.977 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:46 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:46.977 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:46 smithi087 ceph-mon[109386]: Deploying daemon prometheus.a on smithi087 2023-10-01T19:42:46.977 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:46 smithi087 ceph-mon[109386]: [01/Oct/2023:19:42:46] ENGINE Bus STARTING 2023-10-01T19:42:46.977 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:46 smithi087 ceph-mon[109386]: [01/Oct/2023:19:42:46] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:42:46.977 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:46 smithi087 ceph-mon[109386]: [01/Oct/2023:19:42:46] ENGINE Bus STARTED 2023-10-01T19:42:46.977 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:46 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:47.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2023-10-01T19:42:47.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[108286]: mgrmap e19: y(active, since 1.25434s), standbys: x 2023-10-01T19:42:47.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[108286]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi057=a;smithi087=b", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:47.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[108286]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:47.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[108286]: Saving service node-exporter spec with placement smithi057=a;smithi087=b;count:2 2023-10-01T19:42:47.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:47.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[108286]: Deploying daemon prometheus.a on smithi087 2023-10-01T19:42:47.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[108286]: [01/Oct/2023:19:42:46] ENGINE Bus STARTING 2023-10-01T19:42:47.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[108286]: [01/Oct/2023:19:42:46] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:42:47.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[108286]: [01/Oct/2023:19:42:46] ENGINE Bus STARTED 2023-10-01T19:42:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:47.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2023-10-01T19:42:47.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[114331]: mgrmap e19: y(active, since 1.25434s), standbys: x 2023-10-01T19:42:47.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[114331]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi057=a;smithi087=b", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:47.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[114331]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:47.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[114331]: Saving service node-exporter spec with placement smithi057=a;smithi087=b;count:2 2023-10-01T19:42:47.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:47.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[114331]: Deploying daemon prometheus.a on smithi087 2023-10-01T19:42:47.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[114331]: [01/Oct/2023:19:42:46] ENGINE Bus STARTING 2023-10-01T19:42:47.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[114331]: [01/Oct/2023:19:42:46] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:42:47.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[114331]: [01/Oct/2023:19:42:46] ENGINE Bus STARTED 2023-10-01T19:42:47.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:46 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:47.654 INFO:teuthology.orchestra.run.smithi087.stdout:Scheduled alertmanager update... 2023-10-01T19:42:47.972 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:47 smithi057 ceph-mon[108286]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2023-10-01T19:42:47.972 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:47 smithi057 ceph-mon[108286]: mgrmap e20: y(active, since 2s), standbys: x 2023-10-01T19:42:47.972 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:47 smithi057 ceph-mon[108286]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:47.972 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:47 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:47.973 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:47 smithi057 ceph-mon[114331]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2023-10-01T19:42:47.973 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:47 smithi057 ceph-mon[114331]: mgrmap e20: y(active, since 2s), standbys: x 2023-10-01T19:42:47.973 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:47 smithi057 ceph-mon[114331]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:47.973 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:47 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:48.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:47 smithi087 ceph-mon[109386]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2023-10-01T19:42:48.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:47 smithi087 ceph-mon[109386]: mgrmap e20: y(active, since 2s), standbys: x 2023-10-01T19:42:48.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:47 smithi087 ceph-mon[109386]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:48.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:47 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:48.152 DEBUG:teuthology.orchestra.run.smithi057:alertmanager.a> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@alertmanager.a.service 2023-10-01T19:42:48.155 INFO:tasks.cephadm:Adding grafana.a on smithi087 2023-10-01T19:42:48.155 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph orch apply grafana '1;smithi087=a' 2023-10-01T19:42:48.190 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:-- Logs begin at Sun 2023-10-01 19:27:22 UTC. -- 2023-10-01T19:42:48.306 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:48 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:42:48] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:42:49.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:49 smithi057 ceph-mon[108286]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi057=a", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:49.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:49 smithi057 ceph-mon[108286]: Saving service alertmanager spec with placement smithi057=a;count:1 2023-10-01T19:42:49.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:49 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:49.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:49 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:49.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:49 smithi057 ceph-mon[114331]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi057=a", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:49.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:49 smithi057 ceph-mon[114331]: Saving service alertmanager spec with placement smithi057=a;count:1 2023-10-01T19:42:49.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:49 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:49.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:49 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:49.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:49 smithi087 ceph-mon[109386]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi057=a", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:49.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:49 smithi087 ceph-mon[109386]: Saving service alertmanager spec with placement smithi057=a;count:1 2023-10-01T19:42:49.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:49 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:49.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:49 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:50.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:50 smithi087 ceph-mon[109386]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:50.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:50 smithi057 ceph-mon[108286]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:50.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:50 smithi057 ceph-mon[114331]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:52.303 INFO:teuthology.orchestra.run.smithi087.stdout:Scheduled grafana update... 2023-10-01T19:42:52.622 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:52 smithi087 ceph-mon[109386]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:52.622 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:52.682 DEBUG:teuthology.orchestra.run.smithi087:grafana.a> sudo journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@grafana.a.service 2023-10-01T19:42:52.684 INFO:tasks.cephadm:Setting up client nodes... 2023-10-01T19:42:52.684 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-01T19:42:52.716 INFO:journalctl@ceph.grafana.a.smithi087.stdout:-- Logs begin at Sun 2023-10-01 19:27:12 UTC. -- 2023-10-01T19:42:52.720 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:52 smithi057 ceph-mon[108286]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:52.720 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:52.720 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:52 smithi057 ceph-mon[114331]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:52.721 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:53.342 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 systemd[1]: Starting Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:42:53.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:53 smithi057 ceph-mon[108286]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi087=a", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:53.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:53 smithi057 ceph-mon[108286]: Saving service grafana spec with placement smithi087=a;count:1 2023-10-01T19:42:53.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:53 smithi057 ceph-mon[114331]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi087=a", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:53.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:53 smithi057 ceph-mon[114331]: Saving service grafana spec with placement smithi087=a;count:1 2023-10-01T19:42:53.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-mon[109386]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi087=a", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:42:53.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-mon[109386]: Saving service grafana spec with placement smithi087=a;count:1 2023-10-01T19:42:53.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.474Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2023-10-01T19:42:53.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.474Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2023-10-01T19:42:53.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.474Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2023-10-01T19:42:53.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.474Z caller=main.go:518 level=info host_details="(Linux 4.18.0-514.el8.x86_64 #1 SMP Tue Sep 19 13:19:58 UTC 2023 x86_64 smithi087 (none))" 2023-10-01T19:42:53.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.474Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2023-10-01T19:42:53.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.474Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2023-10-01T19:42:53.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.475Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2023-10-01T19:42:53.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.476Z caller=main.go:923 level=info msg="Starting TSDB ..." 2023-10-01T19:42:53.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.477Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2023-10-01T19:42:53.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 bash[127660]: 0365ce2de790dfdfba30174381997be6801680739963cfcc7bb6ca996b56a086 2023-10-01T19:42:53.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.478Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2023-10-01T19:42:53.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.478Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.694µs 2023-10-01T19:42:53.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.478Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2023-10-01T19:42:53.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.478Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2023-10-01T19:42:53.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.478Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=21.226µs wal_replay_duration=113.197µs total_replay_duration=149.509µs 2023-10-01T19:42:53.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.479Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2023-10-01T19:42:53.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.479Z caller=main.go:947 level=info msg="TSDB started" 2023-10-01T19:42:53.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.479Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2023-10-01T19:42:53.885 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 systemd[1]: Started Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:42:53.885 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.496Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.968544ms db_storage=650ns remote_storage=1.395µs web_handler=413ns query_engine=514ns scrape=395.076µs scrape_sd=17.212µs notify=1.021µs notify_sd=1.955µs rules=16.388577ms 2023-10-01T19:42:53.885 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:42:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:42:53.496Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2023-10-01T19:42:54.226 INFO:teuthology.orchestra.run.smithi057.stdout:[client.0] 2023-10-01T19:42:54.226 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQC+yxll6Gi/DBAAU1JYZCrpcT3LF/aN1U5ftQ== 2023-10-01T19:42:54.599 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-01T19:42:54.599 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-01T19:42:54.600 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-01T19:42:54.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: pgmap v7: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:54.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:54.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:54.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: pgmap v8: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: pgmap v9: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:42:54.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:42:54.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:42:54.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.87:9095"}]: dispatch 2023-10-01T19:42:54.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.87:9095"}]: dispatch 2023-10-01T19:42:54.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:54.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:54.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:54.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: pgmap v10: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: pgmap v11: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:42:54.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:54.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:54.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:54.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: pgmap v12: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: pgmap v13: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/4170804067' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-01T19:42:54.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-01T19:42:54.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:54 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-01T19:42:54.692 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-01T19:42:54.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: pgmap v7: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:54.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:54.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:54.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: pgmap v8: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: pgmap v9: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:42:54.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:42:54.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:42:54.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.87:9095"}]: dispatch 2023-10-01T19:42:54.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.87:9095"}]: dispatch 2023-10-01T19:42:54.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:54.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:54.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:54.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: pgmap v10: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: pgmap v11: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:42:54.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:54.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:54.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:54.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: pgmap v12: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: pgmap v13: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/4170804067' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-01T19:42:54.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-01T19:42:54.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-01T19:42:54.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: pgmap v7: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:54.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:54.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:54.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: pgmap v8: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: pgmap v9: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:42:54.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:42:54.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:42:54.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.87:9095"}]: dispatch 2023-10-01T19:42:54.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.87:9095"}]: dispatch 2023-10-01T19:42:54.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: pgmap v10: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: pgmap v11: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: pgmap v12: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: pgmap v13: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:54.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:54.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/4170804067' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-01T19:42:54.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-01T19:42:54.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:54 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-01T19:42:55.774 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:55 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:42:55.774 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:55 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:55.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:55 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:42:55.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:55 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:55.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:55 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:42:55.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:55 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:56.776 INFO:teuthology.orchestra.run.smithi087.stdout:[client.1] 2023-10-01T19:42:56.776 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDAyxllvrcpLRAAsh+iKtTLlTifT8MIOGYL2Q== 2023-10-01T19:42:56.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:56 smithi087 ceph-mon[109386]: pgmap v14: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:56.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:56 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:56.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:42:56] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:42:56.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:56 smithi057 ceph-mon[114331]: pgmap v14: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:56.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:56 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:56 smithi057 ceph-mon[108286]: pgmap v14: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:56 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:57.274 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-10-01T19:42:57.274 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2023-10-01T19:42:57.274 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-01T19:42:57.317 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-01T19:42:57.317 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-01T19:42:57.317 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-01T19:42:57.346 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:42:57] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:42:57.597 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:42:57.605 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[108286]: Detected new or changed devices on smithi087 2023-10-01T19:42:57.606 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[108286]: from='client.? 172.21.15.87:0/1992779473' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-01T19:42:57.606 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-01T19:42:57.606 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-01T19:42:57.606 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:57.606 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:57.606 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:57.606 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:57.606 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:57.606 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:57.607 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[114331]: Detected new or changed devices on smithi087 2023-10-01T19:42:57.607 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[114331]: from='client.? 172.21.15.87:0/1992779473' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-01T19:42:57.607 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-01T19:42:57.607 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-01T19:42:57.607 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:57.607 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:57.607 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:57.607 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:57.608 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:57.608 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:57 smithi087 ceph-mon[109386]: Detected new or changed devices on smithi087 2023-10-01T19:42:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:57 smithi087 ceph-mon[109386]: from='client.? 172.21.15.87:0/1992779473' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-01T19:42:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:57 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-01T19:42:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:57 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-01T19:42:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:42:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:42:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:42:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:42:58.171 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:42:57 smithi057 bash[132441]: Trying to pull quay.io/prometheus/node-exporter:v1.3.1... 2023-10-01T19:42:58.595 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:42:58 smithi057 bash[132441]: Getting image source signatures 2023-10-01T19:42:58.595 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:42:58 smithi057 bash[132441]: Copying blob sha256:b45d31ee2d7f9f452678a85b0c837c29e12089f31ee8dbac6c8c24dfa4054a30 2023-10-01T19:42:58.595 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:42:58 smithi057 bash[132441]: Copying blob sha256:aa2a8d90b84cb2a9c422e7005cd166a008ccf22ef5d7d4f07128478585ce35ea 2023-10-01T19:42:58.595 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:42:58 smithi057 bash[132441]: Copying blob sha256:b5db1e299295edf3005515ab7879c1df64a33c185d3a7a23aa4dcaa17d26f7b3 2023-10-01T19:42:58.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:58 smithi087 ceph-mon[109386]: pgmap v15: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:58.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:58 smithi087 ceph-mon[109386]: Deploying daemon node-exporter.a on smithi057 2023-10-01T19:42:58.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:58 smithi087 ceph-mon[109386]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2023-10-01T19:42:58.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:58 smithi087 ceph-mon[109386]: Cluster is now healthy 2023-10-01T19:42:58.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:58 smithi057 ceph-mon[108286]: pgmap v15: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:58.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:58 smithi057 ceph-mon[108286]: Deploying daemon node-exporter.a on smithi057 2023-10-01T19:42:58.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:58 smithi057 ceph-mon[108286]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2023-10-01T19:42:58.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:58 smithi057 ceph-mon[108286]: Cluster is now healthy 2023-10-01T19:42:58.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:58 smithi057 ceph-mon[114331]: pgmap v15: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:42:58.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:58 smithi057 ceph-mon[114331]: Deploying daemon node-exporter.a on smithi057 2023-10-01T19:42:58.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:58 smithi057 ceph-mon[114331]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2023-10-01T19:42:58.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:58 smithi057 ceph-mon[114331]: Cluster is now healthy 2023-10-01T19:42:58.938 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:42:59.838 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:42:59 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2814970577' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-01T19:43:00.103 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:42:59 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2814970577' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-01T19:43:00.103 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:42:59 smithi057 bash[132441]: Copying config sha256:1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 2023-10-01T19:43:00.103 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:42:59 smithi057 bash[132441]: Writing manifest to image destination 2023-10-01T19:43:00.104 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":20,"active_gid":24343,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":2722579079},{"type":"v1","addr":"172.21.15.57:6801","nonce":2722579079}]},"active_addr":"172.21.15.57:6801/2722579079","active_change":"2023-10-01T19:42:44.433620+0000","active_mgr_features":4540138303579357183,"available":true,"standbys":[{"gid":14442,"name":"x","mgr_features":4540138303579357183,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.57:8443/","prometheus":"http://172.21.15.57:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":54,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":286188666}]},{"addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":418424420}]},{"addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":2843362731}]},{"addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":560615295}]}]}} 2023-10-01T19:43:00.287 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-01T19:43:00.287 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-01T19:43:00.287 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-01T19:43:00.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:42:59 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2814970577' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-01T19:43:00.559 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:00.608 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.356Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2023-10-01T19:43:00.608 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.356Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2023-10-01T19:43:00.608 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.357Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2023-10-01T19:43:00.608 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.357Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2023-10-01T19:43:00.608 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2023-10-01T19:43:00.608 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=arp 2023-10-01T19:43:00.608 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=bcache 2023-10-01T19:43:00.608 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=bonding 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=btrfs 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=conntrack 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=cpu 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=cpufreq 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=diskstats 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=dmi 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=edac 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=entropy 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=fibrechannel 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=filefd 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=filesystem 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=hwmon 2023-10-01T19:43:00.609 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=infiniband 2023-10-01T19:43:00.610 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=ipvs 2023-10-01T19:43:00.610 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=loadavg 2023-10-01T19:43:00.610 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=mdadm 2023-10-01T19:43:00.610 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=meminfo 2023-10-01T19:43:00.610 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=netclass 2023-10-01T19:43:00.610 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=netdev 2023-10-01T19:43:00.610 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=netstat 2023-10-01T19:43:00.610 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=nfs 2023-10-01T19:43:00.610 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=nfsd 2023-10-01T19:43:00.610 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=nvme 2023-10-01T19:43:00.610 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=os 2023-10-01T19:43:00.610 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2023-10-01T19:43:00.611 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=pressure 2023-10-01T19:43:00.611 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=rapl 2023-10-01T19:43:00.611 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=schedstat 2023-10-01T19:43:00.611 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=sockstat 2023-10-01T19:43:00.611 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=softnet 2023-10-01T19:43:00.611 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=stat 2023-10-01T19:43:00.611 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=tapestats 2023-10-01T19:43:00.611 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=textfile 2023-10-01T19:43:00.611 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=thermal_zone 2023-10-01T19:43:00.611 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=time 2023-10-01T19:43:00.611 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=udp_queues 2023-10-01T19:43:00.611 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=uname 2023-10-01T19:43:00.612 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=vmstat 2023-10-01T19:43:00.612 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=xfs 2023-10-01T19:43:00.612 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:115 level=info collector=zfs 2023-10-01T19:43:00.612 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.358Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2023-10-01T19:43:00.612 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[132608]: ts=2023-10-01T19:43:00.359Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2023-10-01T19:43:00.612 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 bash[132441]: efd276ccf21b4ccc963ef0f532a3b5b12f1095758204bc4b87afc94edf4eebed 2023-10-01T19:43:00.612 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:43:00 smithi057 systemd[1]: Started Ceph node-exporter.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:43:00.767 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:00 smithi087 ceph-mon[109386]: pgmap v16: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:00.767 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:00 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:00.902 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-mon[108286]: pgmap v16: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:00.902 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:00.902 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-mon[114331]: pgmap v16: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:00.902 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:00 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:01.133 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:01 smithi087 systemd[1]: Starting Ceph node-exporter.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:43:01.630 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:01 smithi087 bash[128681]: Trying to pull quay.io/prometheus/node-exporter:v1.3.1... 2023-10-01T19:43:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:43:01.816 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":54,"fsid":"0f6d5dc0-6092-11ee-8db5-212e2dc638e7","created":"2023-10-01T19:38:40.968670+0000","modified":"2023-10-01T19:42:44.433032+0000","last_up_change":"2023-10-01T19:42:29.672336+0000","last_in_change":"2023-10-01T19:42:14.852777+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-01T19:41:08.148780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"8c7f9d93-03f1-48e1-9ac3-eab5bbd20afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3459964208},{"type":"v1","addr":"172.21.15.57:6803","nonce":3459964208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3459964208},{"type":"v1","addr":"172.21.15.57:6805","nonce":3459964208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3459964208},{"type":"v1","addr":"172.21.15.57:6809","nonce":3459964208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3459964208},{"type":"v1","addr":"172.21.15.57:6807","nonce":3459964208}]},"public_addr":"172.21.15.57:6803/3459964208","cluster_addr":"172.21.15.57:6805/3459964208","heartbeat_back_addr":"172.21.15.57:6809/3459964208","heartbeat_front_addr":"172.21.15.57:6807/3459964208","state":["exists","up"]},{"osd":1,"uuid":"2272344b-6026-4653-ab79-e9c746993364","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":3617577730},{"type":"v1","addr":"172.21.15.57:6811","nonce":3617577730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":3617577730},{"type":"v1","addr":"172.21.15.57:6813","nonce":3617577730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":3617577730},{"type":"v1","addr":"172.21.15.57:6817","nonce":3617577730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":3617577730},{"type":"v1","addr":"172.21.15.57:6815","nonce":3617577730}]},"public_addr":"172.21.15.57:6811/3617577730","cluster_addr":"172.21.15.57:6813/3617577730","heartbeat_back_addr":"172.21.15.57:6817/3617577730","heartbeat_front_addr":"172.21.15.57:6815/3617577730","state":["exists","up"]},{"osd":2,"uuid":"63bc952e-d249-4ae3-a4cb-a50f415d9beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2187972944},{"type":"v1","addr":"172.21.15.57:6819","nonce":2187972944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2187972944},{"type":"v1","addr":"172.21.15.57:6821","nonce":2187972944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2187972944},{"type":"v1","addr":"172.21.15.57:6825","nonce":2187972944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2187972944},{"type":"v1","addr":"172.21.15.57:6823","nonce":2187972944}]},"public_addr":"172.21.15.57:6819/2187972944","cluster_addr":"172.21.15.57:6821/2187972944","heartbeat_back_addr":"172.21.15.57:6825/2187972944","heartbeat_front_addr":"172.21.15.57:6823/2187972944","state":["exists","up"]},{"osd":3,"uuid":"039ec24f-b99e-4248-b0d9-b9e61bfd8833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1040473437},{"type":"v1","addr":"172.21.15.57:6827","nonce":1040473437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1040473437},{"type":"v1","addr":"172.21.15.57:6829","nonce":1040473437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1040473437},{"type":"v1","addr":"172.21.15.57:6833","nonce":1040473437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1040473437},{"type":"v1","addr":"172.21.15.57:6831","nonce":1040473437}]},"public_addr":"172.21.15.57:6827/1040473437","cluster_addr":"172.21.15.57:6829/1040473437","heartbeat_back_addr":"172.21.15.57:6833/1040473437","heartbeat_front_addr":"172.21.15.57:6831/1040473437","state":["exists","up"]},{"osd":4,"uuid":"709bab5f-07ff-4bf9-9c9c-942bbc4901a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2800415383},{"type":"v1","addr":"172.21.15.87:6801","nonce":2800415383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":2800415383},{"type":"v1","addr":"172.21.15.87:6803","nonce":2800415383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":2800415383},{"type":"v1","addr":"172.21.15.87:6807","nonce":2800415383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":2800415383},{"type":"v1","addr":"172.21.15.87:6805","nonce":2800415383}]},"public_addr":"172.21.15.87:6801/2800415383","cluster_addr":"172.21.15.87:6803/2800415383","heartbeat_back_addr":"172.21.15.87:6807/2800415383","heartbeat_front_addr":"172.21.15.87:6805/2800415383","state":["exists","up"]},{"osd":5,"uuid":"839f46cd-0e7c-4f51-b784-3c85cc3dde2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3413643271},{"type":"v1","addr":"172.21.15.87:6809","nonce":3413643271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3413643271},{"type":"v1","addr":"172.21.15.87:6811","nonce":3413643271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3413643271},{"type":"v1","addr":"172.21.15.87:6815","nonce":3413643271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3413643271},{"type":"v1","addr":"172.21.15.87:6813","nonce":3413643271}]},"public_addr":"172.21.15.87:6809/3413643271","cluster_addr":"172.21.15.87:6811/3413643271","heartbeat_back_addr":"172.21.15.87:6815/3413643271","heartbeat_front_addr":"172.21.15.87:6813/3413643271","state":["exists","up"]},{"osd":6,"uuid":"d2d554e8-b41d-4ddc-803d-6ad5de2c4b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1596710583},{"type":"v1","addr":"172.21.15.87:6817","nonce":1596710583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1596710583},{"type":"v1","addr":"172.21.15.87:6819","nonce":1596710583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1596710583},{"type":"v1","addr":"172.21.15.87:6823","nonce":1596710583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1596710583},{"type":"v1","addr":"172.21.15.87:6821","nonce":1596710583}]},"public_addr":"172.21.15.87:6817/1596710583","cluster_addr":"172.21.15.87:6819/1596710583","heartbeat_back_addr":"172.21.15.87:6823/1596710583","heartbeat_front_addr":"172.21.15.87:6821/1596710583","state":["exists","up"]},{"osd":7,"uuid":"ccdaed0b-f228-491c-a150-5853328e7361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1585249784},{"type":"v1","addr":"172.21.15.87:6825","nonce":1585249784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1585249784},{"type":"v1","addr":"172.21.15.87:6827","nonce":1585249784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1585249784},{"type":"v1","addr":"172.21.15.87:6831","nonce":1585249784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1585249784},{"type":"v1","addr":"172.21.15.87:6829","nonce":1585249784}]},"public_addr":"172.21.15.87:6825/1585249784","cluster_addr":"172.21.15.87:6827/1585249784","heartbeat_back_addr":"172.21.15.87:6831/1585249784","heartbeat_front_addr":"172.21.15.87:6829/1585249784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:40:29.458860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:40:47.602237+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:41:05.611067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:41:23.644060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:41:39.539090+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:41:55.722558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:42:11.880707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:42:28.220193+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:6800/2547329209":"2023-10-02T19:42:44.432981+0000","172.21.15.57:0/1285618755":"2023-10-02T19:42:44.432981+0000","172.21.15.57:0/3512058905":"2023-10-02T19:42:44.432981+0000","172.21.15.57:0/1623382278":"2023-10-02T19:42:44.432981+0000","172.21.15.57:0/427084745":"2023-10-02T19:39:27.540639+0000","172.21.15.57:0/589184120":"2023-10-02T19:39:27.540639+0000","172.21.15.57:0/19155294":"2023-10-02T19:39:27.540639+0000","172.21.15.57:6800/3618988359":"2023-10-02T19:39:27.540639+0000","172.21.15.57:0/1453123188":"2023-10-02T19:39:05.183540+0000","172.21.15.57:0/34608214":"2023-10-02T19:39:05.183540+0000","172.21.15.57:0/2189167615":"2023-10-02T19:42:44.432981+0000","172.21.15.57:6801/3618988359":"2023-10-02T19:39:27.540639+0000","172.21.15.57:6800/1407753143":"2023-10-02T19:39:05.183540+0000","172.21.15.57:6801/2547329209":"2023-10-02T19:42:44.432981+0000","172.21.15.57:0/69774414":"2023-10-02T19:39:05.183540+0000","172.21.15.57:6801/1407753143":"2023-10-02T19:39:05.183540+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-01T19:43:01.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:01 smithi087 ceph-mon[109386]: Deploying daemon node-exporter.b on smithi087 2023-10-01T19:43:01.882 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:01 smithi087 bash[128681]: Getting image source signatures 2023-10-01T19:43:01.883 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:01 smithi087 bash[128681]: Copying blob sha256:b5db1e299295edf3005515ab7879c1df64a33c185d3a7a23aa4dcaa17d26f7b3 2023-10-01T19:43:01.883 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:01 smithi087 bash[128681]: Copying blob sha256:aa2a8d90b84cb2a9c422e7005cd166a008ccf22ef5d7d4f07128478585ce35ea 2023-10-01T19:43:01.883 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:01 smithi087 bash[128681]: Copying blob sha256:b45d31ee2d7f9f452678a85b0c837c29e12089f31ee8dbac6c8c24dfa4054a30 2023-10-01T19:43:01.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:01 smithi057 ceph-mon[108286]: Deploying daemon node-exporter.b on smithi087 2023-10-01T19:43:01.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:01 smithi057 ceph-mon[114331]: Deploying daemon node-exporter.b on smithi087 2023-10-01T19:43:02.219 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-01T19:43:02.220 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-01T19:43:02.479 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:02.919 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:02 smithi057 ceph-mon[108286]: pgmap v17: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:02.919 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:02 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3958864206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-01T19:43:02.920 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:02 smithi057 ceph-mon[114331]: pgmap v17: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:02.920 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:02 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3958864206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-01T19:43:02.985 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:02 smithi087 ceph-mon[109386]: pgmap v17: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:02.985 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:02 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3958864206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-01T19:43:03.383 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:02 smithi087 bash[128681]: Copying config sha256:1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 2023-10-01T19:43:03.383 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 bash[128681]: Writing manifest to image destination 2023-10-01T19:43:03.661 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.551Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2023-10-01T19:43:03.661 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.552Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2023-10-01T19:43:03.661 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.552Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2023-10-01T19:43:03.661 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.552Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2023-10-01T19:43:03.661 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2023-10-01T19:43:03.662 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=arp 2023-10-01T19:43:03.662 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=bcache 2023-10-01T19:43:03.662 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=bonding 2023-10-01T19:43:03.662 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=btrfs 2023-10-01T19:43:03.662 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=conntrack 2023-10-01T19:43:03.662 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=cpu 2023-10-01T19:43:03.662 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=cpufreq 2023-10-01T19:43:03.662 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=diskstats 2023-10-01T19:43:03.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=dmi 2023-10-01T19:43:03.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=edac 2023-10-01T19:43:03.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=entropy 2023-10-01T19:43:03.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=fibrechannel 2023-10-01T19:43:03.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=filefd 2023-10-01T19:43:03.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=filesystem 2023-10-01T19:43:03.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=hwmon 2023-10-01T19:43:03.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=infiniband 2023-10-01T19:43:03.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=ipvs 2023-10-01T19:43:03.664 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=loadavg 2023-10-01T19:43:03.664 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=mdadm 2023-10-01T19:43:03.664 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=meminfo 2023-10-01T19:43:03.664 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=netclass 2023-10-01T19:43:03.664 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=netdev 2023-10-01T19:43:03.664 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=netstat 2023-10-01T19:43:03.664 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=nfs 2023-10-01T19:43:03.664 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=nfsd 2023-10-01T19:43:03.665 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=nvme 2023-10-01T19:43:03.665 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=os 2023-10-01T19:43:03.665 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2023-10-01T19:43:03.665 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=pressure 2023-10-01T19:43:03.665 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=rapl 2023-10-01T19:43:03.665 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=schedstat 2023-10-01T19:43:03.665 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=sockstat 2023-10-01T19:43:03.665 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=softnet 2023-10-01T19:43:03.666 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=stat 2023-10-01T19:43:03.666 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=tapestats 2023-10-01T19:43:03.666 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=textfile 2023-10-01T19:43:03.666 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=thermal_zone 2023-10-01T19:43:03.666 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=time 2023-10-01T19:43:03.666 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=udp_queues 2023-10-01T19:43:03.666 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=uname 2023-10-01T19:43:03.666 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=vmstat 2023-10-01T19:43:03.666 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=xfs 2023-10-01T19:43:03.667 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:115 level=info collector=zfs 2023-10-01T19:43:03.667 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2023-10-01T19:43:03.667 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[128732]: ts=2023-10-01T19:43:03.553Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2023-10-01T19:43:03.667 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 bash[128681]: 6d094569ea475370b5bd7f5ceddc1c9fea8156c62df72aecb35baf731cc9d94b 2023-10-01T19:43:03.667 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:43:03 smithi087 systemd[1]: Started Ceph node-exporter.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:43:03.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:03 smithi057 ceph-mon[108286]: pgmap v18: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:03.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:03 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:03.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:03 smithi057 ceph-mon[114331]: pgmap v18: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:03.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:03 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:43:04.121 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":54,"fsid":"0f6d5dc0-6092-11ee-8db5-212e2dc638e7","created":"2023-10-01T19:38:40.968670+0000","modified":"2023-10-01T19:42:44.433032+0000","last_up_change":"2023-10-01T19:42:29.672336+0000","last_in_change":"2023-10-01T19:42:14.852777+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-01T19:41:08.148780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"8c7f9d93-03f1-48e1-9ac3-eab5bbd20afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3459964208},{"type":"v1","addr":"172.21.15.57:6803","nonce":3459964208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3459964208},{"type":"v1","addr":"172.21.15.57:6805","nonce":3459964208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3459964208},{"type":"v1","addr":"172.21.15.57:6809","nonce":3459964208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3459964208},{"type":"v1","addr":"172.21.15.57:6807","nonce":3459964208}]},"public_addr":"172.21.15.57:6803/3459964208","cluster_addr":"172.21.15.57:6805/3459964208","heartbeat_back_addr":"172.21.15.57:6809/3459964208","heartbeat_front_addr":"172.21.15.57:6807/3459964208","state":["exists","up"]},{"osd":1,"uuid":"2272344b-6026-4653-ab79-e9c746993364","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":3617577730},{"type":"v1","addr":"172.21.15.57:6811","nonce":3617577730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":3617577730},{"type":"v1","addr":"172.21.15.57:6813","nonce":3617577730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":3617577730},{"type":"v1","addr":"172.21.15.57:6817","nonce":3617577730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":3617577730},{"type":"v1","addr":"172.21.15.57:6815","nonce":3617577730}]},"public_addr":"172.21.15.57:6811/3617577730","cluster_addr":"172.21.15.57:6813/3617577730","heartbeat_back_addr":"172.21.15.57:6817/3617577730","heartbeat_front_addr":"172.21.15.57:6815/3617577730","state":["exists","up"]},{"osd":2,"uuid":"63bc952e-d249-4ae3-a4cb-a50f415d9beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2187972944},{"type":"v1","addr":"172.21.15.57:6819","nonce":2187972944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2187972944},{"type":"v1","addr":"172.21.15.57:6821","nonce":2187972944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2187972944},{"type":"v1","addr":"172.21.15.57:6825","nonce":2187972944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2187972944},{"type":"v1","addr":"172.21.15.57:6823","nonce":2187972944}]},"public_addr":"172.21.15.57:6819/2187972944","cluster_addr":"172.21.15.57:6821/2187972944","heartbeat_back_addr":"172.21.15.57:6825/2187972944","heartbeat_front_addr":"172.21.15.57:6823/2187972944","state":["exists","up"]},{"osd":3,"uuid":"039ec24f-b99e-4248-b0d9-b9e61bfd8833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1040473437},{"type":"v1","addr":"172.21.15.57:6827","nonce":1040473437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1040473437},{"type":"v1","addr":"172.21.15.57:6829","nonce":1040473437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1040473437},{"type":"v1","addr":"172.21.15.57:6833","nonce":1040473437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1040473437},{"type":"v1","addr":"172.21.15.57:6831","nonce":1040473437}]},"public_addr":"172.21.15.57:6827/1040473437","cluster_addr":"172.21.15.57:6829/1040473437","heartbeat_back_addr":"172.21.15.57:6833/1040473437","heartbeat_front_addr":"172.21.15.57:6831/1040473437","state":["exists","up"]},{"osd":4,"uuid":"709bab5f-07ff-4bf9-9c9c-942bbc4901a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2800415383},{"type":"v1","addr":"172.21.15.87:6801","nonce":2800415383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":2800415383},{"type":"v1","addr":"172.21.15.87:6803","nonce":2800415383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":2800415383},{"type":"v1","addr":"172.21.15.87:6807","nonce":2800415383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":2800415383},{"type":"v1","addr":"172.21.15.87:6805","nonce":2800415383}]},"public_addr":"172.21.15.87:6801/2800415383","cluster_addr":"172.21.15.87:6803/2800415383","heartbeat_back_addr":"172.21.15.87:6807/2800415383","heartbeat_front_addr":"172.21.15.87:6805/2800415383","state":["exists","up"]},{"osd":5,"uuid":"839f46cd-0e7c-4f51-b784-3c85cc3dde2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3413643271},{"type":"v1","addr":"172.21.15.87:6809","nonce":3413643271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3413643271},{"type":"v1","addr":"172.21.15.87:6811","nonce":3413643271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3413643271},{"type":"v1","addr":"172.21.15.87:6815","nonce":3413643271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3413643271},{"type":"v1","addr":"172.21.15.87:6813","nonce":3413643271}]},"public_addr":"172.21.15.87:6809/3413643271","cluster_addr":"172.21.15.87:6811/3413643271","heartbeat_back_addr":"172.21.15.87:6815/3413643271","heartbeat_front_addr":"172.21.15.87:6813/3413643271","state":["exists","up"]},{"osd":6,"uuid":"d2d554e8-b41d-4ddc-803d-6ad5de2c4b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1596710583},{"type":"v1","addr":"172.21.15.87:6817","nonce":1596710583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1596710583},{"type":"v1","addr":"172.21.15.87:6819","nonce":1596710583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1596710583},{"type":"v1","addr":"172.21.15.87:6823","nonce":1596710583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1596710583},{"type":"v1","addr":"172.21.15.87:6821","nonce":1596710583}]},"public_addr":"172.21.15.87:6817/1596710583","cluster_addr":"172.21.15.87:6819/1596710583","heartbeat_back_addr":"172.21.15.87:6823/1596710583","heartbeat_front_addr":"172.21.15.87:6821/1596710583","state":["exists","up"]},{"osd":7,"uuid":"ccdaed0b-f228-491c-a150-5853328e7361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1585249784},{"type":"v1","addr":"172.21.15.87:6825","nonce":1585249784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1585249784},{"type":"v1","addr":"172.21.15.87:6827","nonce":1585249784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1585249784},{"type":"v1","addr":"172.21.15.87:6831","nonce":1585249784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1585249784},{"type":"v1","addr":"172.21.15.87:6829","nonce":1585249784}]},"public_addr":"172.21.15.87:6825/1585249784","cluster_addr":"172.21.15.87:6827/1585249784","heartbeat_back_addr":"172.21.15.87:6831/1585249784","heartbeat_front_addr":"172.21.15.87:6829/1585249784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:40:29.458860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:40:47.602237+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:41:05.611067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:41:23.644060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:41:39.539090+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:41:55.722558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:42:11.880707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2023-10-01T19:42:28.220193+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:6800/2547329209":"2023-10-02T19:42:44.432981+0000","172.21.15.57:0/1285618755":"2023-10-02T19:42:44.432981+0000","172.21.15.57:0/3512058905":"2023-10-02T19:42:44.432981+0000","172.21.15.57:0/1623382278":"2023-10-02T19:42:44.432981+0000","172.21.15.57:0/427084745":"2023-10-02T19:39:27.540639+0000","172.21.15.57:0/589184120":"2023-10-02T19:39:27.540639+0000","172.21.15.57:0/19155294":"2023-10-02T19:39:27.540639+0000","172.21.15.57:6800/3618988359":"2023-10-02T19:39:27.540639+0000","172.21.15.57:0/1453123188":"2023-10-02T19:39:05.183540+0000","172.21.15.57:0/34608214":"2023-10-02T19:39:05.183540+0000","172.21.15.57:0/2189167615":"2023-10-02T19:42:44.432981+0000","172.21.15.57:6801/3618988359":"2023-10-02T19:39:27.540639+0000","172.21.15.57:6800/1407753143":"2023-10-02T19:39:05.183540+0000","172.21.15.57:6801/2547329209":"2023-10-02T19:42:44.432981+0000","172.21.15.57:0/69774414":"2023-10-02T19:39:05.183540+0000","172.21.15.57:6801/1407753143":"2023-10-02T19:39:05.183540+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-01T19:43:04.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-mon[109386]: pgmap v18: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:04.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:03 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:04.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:04 smithi087 ceph-mon[109386]: Deploying daemon alertmanager.a on smithi057 2023-10-01T19:43:04.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:04 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2103473347' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-01T19:43:04.726 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:04 smithi057 ceph-mon[108286]: Deploying daemon alertmanager.a on smithi057 2023-10-01T19:43:04.726 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:04 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2103473347' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-01T19:43:04.726 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:04 smithi057 ceph-mon[114331]: Deploying daemon alertmanager.a on smithi057 2023-10-01T19:43:04.726 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:04 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2103473347' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-01T19:43:04.727 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-01T19:43:04.728 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-01T19:43:04.728 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-01T19:43:04.729 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-01T19:43:04.729 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-01T19:43:04.729 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-01T19:43:04.730 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-01T19:43:04.730 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-01T19:43:05.516 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:05.529 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:05.688 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:05.994 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:05 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:05.994 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:05 smithi057 ceph-mon[108286]: pgmap v19: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:05.995 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:05 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:05.995 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:05 smithi057 ceph-mon[114331]: pgmap v19: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:06.073 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:05 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:06.073 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:05 smithi087 ceph-mon[109386]: pgmap v19: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:06.690 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:07.104 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:07.105 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:07.109 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:07.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:06 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:43:06] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:07.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:06 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:43:06] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:07.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:07.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:07.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:07.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:07 smithi057 ceph-mon[108286]: pgmap v20: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:07.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:07.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:07.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:07.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:07 smithi057 ceph-mon[114331]: pgmap v20: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:08.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:07 smithi087 ceph-mon[109386]: pgmap v20: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:08.603 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:08 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:43:08] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:09.558 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:09 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:09.558 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:09 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:09.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:09 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:10.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:10 smithi057 ceph-mon[108286]: pgmap v21: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:10.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:10 smithi057 ceph-mon[114331]: pgmap v21: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:10.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:10 smithi087 ceph-mon[109386]: pgmap v21: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:11.523 INFO:teuthology.orchestra.run.smithi057.stdout:180388626445 2023-10-01T19:43:11.524 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-01T19:43:12.591 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:12 smithi057 ceph-mon[114331]: pgmap v22: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:12.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:12 smithi057 ceph-mon[108286]: pgmap v22: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:12.613 INFO:teuthology.orchestra.run.smithi057.stdout:34359738402 2023-10-01T19:43:12.613 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-01T19:43:12.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:12 smithi087 ceph-mon[109386]: pgmap v22: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:13.047 INFO:teuthology.orchestra.run.smithi057.stdout:55834574877 2023-10-01T19:43:13.047 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-01T19:43:13.185 INFO:teuthology.orchestra.run.smithi057.stdout:120259084309 2023-10-01T19:43:13.185 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-01T19:43:13.541 INFO:teuthology.orchestra.run.smithi057.stdout:210453397514 2023-10-01T19:43:13.541 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-01T19:43:13.670 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:13.696 INFO:teuthology.orchestra.run.smithi057.stdout:150323855375 2023-10-01T19:43:13.696 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-01T19:43:13.869 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:14.277 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:14.581 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:14 smithi057 ceph-mon[114331]: pgmap v23: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:14.581 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:14 smithi057 ceph-mon[108286]: pgmap v23: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:14.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:14 smithi087 ceph-mon[109386]: pgmap v23: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:15.823 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:15 smithi057 ceph-mon[108286]: pgmap v24: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:15.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:15 smithi057 ceph-mon[114331]: pgmap v24: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:15.825 INFO:teuthology.orchestra.run.smithi057.stdout:103079215128 2023-10-01T19:43:15.825 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-01T19:43:15.957 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:16.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:15 smithi087 ceph-mon[109386]: pgmap v24: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:16.338 INFO:teuthology.orchestra.run.smithi057.stdout:77309411354 2023-10-01T19:43:16.338 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-01T19:43:17.050 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:17.205 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:34359738403 2023-10-01T19:43:17.519 INFO:teuthology.orchestra.run.smithi057.stdout:180388626447 2023-10-01T19:43:17.582 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:17.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:17 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:43:17] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:18.022 INFO:teuthology.orchestra.run.smithi057.stdout:120259084310 2023-10-01T19:43:18.307 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-mon[114331]: pgmap v25: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:18.307 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/103063780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-01T19:43:18.307 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2286586791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-01T19:43:18.307 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:18.307 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:18.307 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2295932007' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-01T19:43:18.307 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-mon[108286]: pgmap v25: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:18.308 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/103063780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-01T19:43:18.308 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2286586791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-01T19:43:18.308 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:18.308 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:18.308 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2295932007' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-01T19:43:18.329 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626445 got 180388626447 for osd.6 2023-10-01T19:43:18.329 DEBUG:teuthology.parallel:result is None 2023-10-01T19:43:18.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:18 smithi087 ceph-mon[109386]: pgmap v25: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:18 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/103063780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-01T19:43:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:18 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2286586791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-01T19:43:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:18 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:18 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:18 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2295932007' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-01T19:43:18.615 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:18.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:18 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[136328]: level=info ts=2023-10-01T19:43:18.695Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2023-10-01T19:43:19.099 INFO:tasks.cephadm.ceph_manager.ceph:need seq 120259084309 got 120259084310 for osd.4 2023-10-01T19:43:19.099 DEBUG:teuthology.parallel:result is None 2023-10-01T19:43:19.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:19.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:19.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:19.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-10-01T19:43:19.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-10-01T19:43:19.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:19.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[108286]: Deploying daemon grafana.a on smithi087 2023-10-01T19:43:19.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[108286]: pgmap v26: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:19.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:19.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:19.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:19.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:19.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-10-01T19:43:19.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-10-01T19:43:19.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:19.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[114331]: Deploying daemon grafana.a on smithi087 2023-10-01T19:43:19.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[114331]: pgmap v26: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:19.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:19 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:20.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:19 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:20.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:19 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:20.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:19 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:20.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:19 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-10-01T19:43:20.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:19 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-10-01T19:43:20.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:19 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:20.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:19 smithi087 ceph-mon[109386]: Deploying daemon grafana.a on smithi087 2023-10-01T19:43:20.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:19 smithi087 ceph-mon[109386]: pgmap v26: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:20.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:19 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:20.249 INFO:teuthology.orchestra.run.smithi057.stdout:103079215129 2023-10-01T19:43:20.914 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:20 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[136328]: level=info ts=2023-10-01T19:43:20.662Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000666733s 2023-10-01T19:43:20.914 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:20 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/730811700' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-01T19:43:20.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:20 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/730811700' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-01T19:43:20.937 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738402 got 34359738403 for osd.0 2023-10-01T19:43:20.937 DEBUG:teuthology.parallel:result is None 2023-10-01T19:43:21.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:20 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/730811700' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-01T19:43:21.198 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215128 got 103079215129 for osd.3 2023-10-01T19:43:21.199 DEBUG:teuthology.parallel:result is None 2023-10-01T19:43:21.583 INFO:teuthology.orchestra.run.smithi057.stdout:150323855376 2023-10-01T19:43:21.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:21 smithi057 ceph-mon[108286]: pgmap v27: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:21.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:21 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2129630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-01T19:43:21.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:21 smithi057 ceph-mon[114331]: pgmap v27: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:21.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:21 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2129630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-01T19:43:22.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:21 smithi087 ceph-mon[109386]: pgmap v27: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:22.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:21 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2129630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-01T19:43:22.151 INFO:teuthology.orchestra.run.smithi057.stdout:55834574879 2023-10-01T19:43:22.364 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855375 got 150323855376 for osd.5 2023-10-01T19:43:22.364 DEBUG:teuthology.parallel:result is None 2023-10-01T19:43:22.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:22 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1929860957' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-01T19:43:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:22 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1929860957' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-01T19:43:22.963 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574877 got 55834574879 for osd.1 2023-10-01T19:43:22.963 DEBUG:teuthology.parallel:result is None 2023-10-01T19:43:23.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:22 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1929860957' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-01T19:43:23.348 INFO:teuthology.orchestra.run.smithi057.stdout:210453397516 2023-10-01T19:43:23.440 INFO:teuthology.orchestra.run.smithi057.stdout:77309411356 2023-10-01T19:43:23.961 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:23 smithi057 ceph-mon[108286]: pgmap v28: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:23.961 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:23 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2946955151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-01T19:43:23.961 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:23 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/39886930' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-01T19:43:23.962 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:23 smithi057 ceph-mon[114331]: pgmap v28: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:23.962 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:23 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2946955151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-01T19:43:23.962 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:23 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/39886930' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-01T19:43:23.963 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411354 got 77309411356 for osd.2 2023-10-01T19:43:23.963 DEBUG:teuthology.parallel:result is None 2023-10-01T19:43:24.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:23 smithi087 ceph-mon[109386]: pgmap v28: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:24.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:23 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2946955151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-01T19:43:24.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:23 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/39886930' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-01T19:43:24.190 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397514 got 210453397516 for osd.7 2023-10-01T19:43:24.190 DEBUG:teuthology.parallel:result is None 2023-10-01T19:43:24.190 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-01T19:43:24.190 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-01T19:43:24.803 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:24.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:43:24] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:25.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:25 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:25.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:25 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:25.889 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:25 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:26.503 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:43:26.505 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-01T19:43:26.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:26 smithi057 ceph-mon[108286]: pgmap v29: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:26.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:26 smithi057 ceph-mon[114331]: pgmap v29: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:26.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:26 smithi087 ceph-mon[109386]: pgmap v29: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:27.415 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:27 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:43:27] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:27.415 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:27 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:43:27] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:27.438 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-10-01T19:43:25.044405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":48212,"kb_used_data":5716,"kb_used_omap":0,"kb_used_meta":42432,"kb_avail":749913004,"statfs":{"total":767960285184,"available":767910916096,"internally_reserved":0,"allocated":5853184,"data_stored":3329309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43450368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002605"},"pg_stats":[{"pgid":"1.0","version":"54'128","reported_seq":266,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-01T19:42:44.765106+0000","last_change":"2023-10-01T19:42:33.777502+0000","last_active":"2023-10-01T19:42:44.765106+0000","last_peered":"2023-10-01T19:42:44.765106+0000","last_clean":"2023-10-01T19:42:44.765106+0000","last_became_active":"2023-10-01T19:42:33.777103+0000","last_became_peered":"2023-10-01T19:42:33.777103+0000","last_unstale":"2023-10-01T19:42:44.765106+0000","last_undegraded":"2023-10-01T19:42:44.765106+0000","last_fullsized":"2023-10-01T19:42:44.765106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-01T19:41:08.520137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-01T19:41:08.520137+0000","last_clean_scrub_stamp":"2023-10-01T19:41:08.520137+0000","objects_scrubbed":0,"log_size":128,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-03T02:48:52.540391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6040,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739112,"statfs":{"total":95995035648,"available":95988850688,"internally_reserved":0,"allocated":1064960,"data_stored":746784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]}]},{"osd":6,"up_from":42,"seq":180388626448,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6104,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739048,"statfs":{"total":95995035648,"available":95988785152,"internally_reserved":0,"allocated":1064960,"data_stored":746784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.5},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.42799999999999999}]},{"osd":1,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.438}]},{"osd":2,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51400000000000001}]},{"osd":3,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47399999999999998}]},{"osd":4,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6224,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738928,"statfs":{"total":95995035648,"available":95988662272,"internally_reserved":0,"allocated":532480,"data_stored":217856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 1 19:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.61099999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.443,"15min":0.44800000000000001},"min":{"1min":0.28799999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60799999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.35399999999999998}]},{"osd":2,"last update":"Sun Oct 1 19:43:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.32100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.44},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.45700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65800000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.47599999999999998}]},{"osd":3,"last update":"Sun Oct 1 19:42:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.67600000000000005}]},{"osd":4,"last update":"Sun Oct 1 19:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.622},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.81799999999999995}]},{"osd":5,"last update":"Sun Oct 1 19:43:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.69599999999999995}]},{"osd":6,"last update":"Sun Oct 1 19:43:13 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738404,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6808,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738344,"statfs":{"total":95995035648,"available":95988064256,"internally_reserved":0,"allocated":1064960,"data_stored":746784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 1 19:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.55500000000000005,"5min":0.623,"15min":0.623},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.432,"5min":0.39800000000000002,"15min":0.39200000000000002},"min":{"1min":0.30399999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.39000000000000001}]},{"osd":2,"last update":"Sun Oct 1 19:43:13 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.437,"15min":0.437},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.31},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.47999999999999998,"15min":0.47099999999999997},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.55100000000000005}]},{"osd":3,"last update":"Sun Oct 1 19:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.36499999999999999}]},{"osd":4,"last update":"Sun Oct 1 19:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.58499999999999996}]},{"osd":5,"last update":"Sun Oct 1 19:42:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.60599999999999998}]},{"osd":6,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.623},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.56899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411356,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6160,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738992,"statfs":{"total":95995035648,"available":95988727808,"internally_reserved":0,"allocated":532480,"data_stored":217856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 1 19:43:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.30099999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.436,"15min":0.436},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.55800000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.33800000000000002}]},{"osd":1,"last update":"Sun Oct 1 19:43:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41899999999999998,"15min":0.41799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55800000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45600000000000002,"15min":0.45300000000000001},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.59399999999999997,"5min":0.629,"15min":0.629},"last":0.38800000000000001}]},{"osd":3,"last update":"Sun Oct 1 19:42:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.40799999999999997}]},{"osd":4,"last update":"Sun Oct 1 19:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.42199999999999999}]},{"osd":5,"last update":"Sun Oct 1 19:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.49199999999999999}]},{"osd":6,"last update":"Sun Oct 1 19:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]}]},{"osd":3,"up_from":24,"seq":103079215130,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5648,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739504,"statfs":{"total":95995035648,"available":95989252096,"internally_reserved":0,"allocated":532480,"data_stored":217856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 1 19:42:26 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.61299999999999999}]},{"osd":1,"last update":"Sun Oct 1 19:42:26 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Sun Oct 1 19:42:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.63}]},{"osd":4,"last update":"Sun Oct 1 19:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":1.026}]},{"osd":5,"last update":"Sun Oct 1 19:43:00 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.83699999999999997}]},{"osd":6,"last update":"Sun Oct 1 19:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":1.056}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":4,"up_from":28,"seq":120259084311,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5648,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739504,"statfs":{"total":95995035648,"available":95989252096,"internally_reserved":0,"allocated":532480,"data_stored":217856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 1 19:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Sun Oct 1 19:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52200000000000002}]},{"osd":2,"last update":"Sun Oct 1 19:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.72399999999999998}]},{"osd":3,"last update":"Sun Oct 1 19:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.61599999999999999}]},{"osd":5,"last update":"Sun Oct 1 19:43:00 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.625}]},{"osd":6,"last update":"Sun Oct 1 19:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":5,"up_from":35,"seq":150323855377,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5580,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739572,"statfs":{"total":95995035648,"available":95989321728,"internally_reserved":0,"allocated":528384,"data_stored":217533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 1 19:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.374},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55900000000000005}]},{"osd":1,"last update":"Sun Oct 1 19:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.497},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.60199999999999998}]},{"osd":2,"last update":"Sun Oct 1 19:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.44500000000000001}]},{"osd":3,"last update":"Sun Oct 1 19:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54500000000000004}]},{"osd":4,"last update":"Sun Oct 1 19:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.45600000000000002}]},{"osd":6,"last update":"Sun Oct 1 19:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.47699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-01T19:43:27.440 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-01T19:43:27.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:27 smithi057 ceph-mon[108286]: from='client.24533 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-01T19:43:27.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:27 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:27.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:27 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:27.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:27 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:27.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:27 smithi057 ceph-mon[114331]: from='client.24533 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-01T19:43:27.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:27 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:27.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:27 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:27.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:27 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:27.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:27 smithi087 ceph-mon[109386]: from='client.24533 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-01T19:43:27.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:27 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:27.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:27 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:27.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:27 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:28.147 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:27 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:43:27] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:28.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:28 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:43:28] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:28.849 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:28 smithi087 ceph-mon[109386]: pgmap v30: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:28.849 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:28 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:28.849 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:28 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:28.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:28 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:28.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:28 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:28.894 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:43:28.896 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-01T19:43:28.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:28 smithi057 ceph-mon[108286]: pgmap v30: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:28.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:28 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:28.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:28 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:28.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:28 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:28.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:28 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:28.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:28 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[136328]: level=info ts=2023-10-01T19:43:28.665Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.0041292s 2023-10-01T19:43:28.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:28 smithi057 ceph-mon[114331]: pgmap v30: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:28.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:28 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:28.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:28 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:28.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:28 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:28.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:28 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:29.262 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-01T19:43:27.044757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":48212,"kb_used_data":5716,"kb_used_omap":0,"kb_used_meta":42432,"kb_avail":749913004,"statfs":{"total":767960285184,"available":767910916096,"internally_reserved":0,"allocated":5853184,"data_stored":3329309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43450368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002593"},"pg_stats":[{"pgid":"1.0","version":"54'128","reported_seq":266,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-01T19:42:44.765106+0000","last_change":"2023-10-01T19:42:33.777502+0000","last_active":"2023-10-01T19:42:44.765106+0000","last_peered":"2023-10-01T19:42:44.765106+0000","last_clean":"2023-10-01T19:42:44.765106+0000","last_became_active":"2023-10-01T19:42:33.777103+0000","last_became_peered":"2023-10-01T19:42:33.777103+0000","last_unstale":"2023-10-01T19:42:44.765106+0000","last_undegraded":"2023-10-01T19:42:44.765106+0000","last_fullsized":"2023-10-01T19:42:44.765106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-01T19:41:08.520137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-01T19:41:08.520137+0000","last_clean_scrub_stamp":"2023-10-01T19:41:08.520137+0000","objects_scrubbed":0,"log_size":128,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-03T02:48:52.540391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6040,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739112,"statfs":{"total":95995035648,"available":95988850688,"internally_reserved":0,"allocated":1064960,"data_stored":746784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]}]},{"osd":6,"up_from":42,"seq":180388626449,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6104,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739048,"statfs":{"total":95995035648,"available":95988785152,"internally_reserved":0,"allocated":1064960,"data_stored":746784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.48999999999999999}]},{"osd":1,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.502},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.43099999999999999}]},{"osd":2,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.441},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.53700000000000003}]},{"osd":4,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.312},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.33000000000000002}]},{"osd":5,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.36499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6224,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738928,"statfs":{"total":95995035648,"available":95988662272,"internally_reserved":0,"allocated":532480,"data_stored":217856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 1 19:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.61099999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.443,"15min":0.44800000000000001},"min":{"1min":0.28799999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60799999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Oct 1 19:43:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.32100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.45700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65800000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.40699999999999997}]},{"osd":3,"last update":"Sun Oct 1 19:42:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.42699999999999999}]},{"osd":4,"last update":"Sun Oct 1 19:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.443}]},{"osd":5,"last update":"Sun Oct 1 19:43:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.51200000000000001}]},{"osd":6,"last update":"Sun Oct 1 19:43:13 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738405,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6808,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738344,"statfs":{"total":95995035648,"available":95988064256,"internally_reserved":0,"allocated":1064960,"data_stored":746784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 1 19:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.55500000000000005,"5min":0.623,"15min":0.623},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.39800000000000002,"15min":0.39200000000000002},"min":{"1min":0.30399999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.377}]},{"osd":2,"last update":"Sun Oct 1 19:43:13 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.437,"15min":0.437},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.47999999999999998,"15min":0.47099999999999997},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.42199999999999999}]},{"osd":3,"last update":"Sun Oct 1 19:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.316},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.49399999999999999}]},{"osd":4,"last update":"Sun Oct 1 19:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Oct 1 19:42:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71199999999999997}]},{"osd":6,"last update":"Sun Oct 1 19:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.55500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411357,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6160,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738992,"statfs":{"total":95995035648,"available":95988727808,"internally_reserved":0,"allocated":532480,"data_stored":217856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 1 19:43:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.30099999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.432,"5min":0.436,"15min":0.436},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.55800000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.42499999999999999}]},{"osd":1,"last update":"Sun Oct 1 19:43:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41899999999999998,"15min":0.41799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55800000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45600000000000002,"15min":0.45300000000000001},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.59399999999999997,"5min":0.629,"15min":0.629},"last":0.55800000000000005}]},{"osd":3,"last update":"Sun Oct 1 19:42:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.58599999999999997}]},{"osd":4,"last update":"Sun Oct 1 19:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61599999999999999}]},{"osd":5,"last update":"Sun Oct 1 19:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.629}]},{"osd":6,"last update":"Sun Oct 1 19:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215131,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5648,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739504,"statfs":{"total":95995035648,"available":95989252096,"internally_reserved":0,"allocated":532480,"data_stored":217856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 1 19:42:26 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.61299999999999999}]},{"osd":1,"last update":"Sun Oct 1 19:42:26 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Sun Oct 1 19:42:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.63}]},{"osd":4,"last update":"Sun Oct 1 19:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":1.026}]},{"osd":5,"last update":"Sun Oct 1 19:43:00 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.83699999999999997}]},{"osd":6,"last update":"Sun Oct 1 19:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":1.056}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":4,"up_from":28,"seq":120259084312,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5648,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739504,"statfs":{"total":95995035648,"available":95989252096,"internally_reserved":0,"allocated":532480,"data_stored":217856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 1 19:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.379},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.47399999999999998}]},{"osd":1,"last update":"Sun Oct 1 19:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.44},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.43099999999999999}]},{"osd":2,"last update":"Sun Oct 1 19:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.41999999999999998}]},{"osd":3,"last update":"Sun Oct 1 19:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.46700000000000003}]},{"osd":5,"last update":"Sun Oct 1 19:43:00 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.32900000000000001}]},{"osd":6,"last update":"Sun Oct 1 19:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.45700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]}]},{"osd":5,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5580,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739572,"statfs":{"total":95995035648,"available":95989321728,"internally_reserved":0,"allocated":528384,"data_stored":217533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 1 19:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.374}]},{"osd":1,"last update":"Sun Oct 1 19:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.495},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.41099999999999998}]},{"osd":2,"last update":"Sun Oct 1 19:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.443},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46700000000000003}]},{"osd":3,"last update":"Sun Oct 1 19:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.72899999999999998}]},{"osd":4,"last update":"Sun Oct 1 19:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.32300000000000001}]},{"osd":6,"last update":"Sun Oct 1 19:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.432},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-01T19:43:29.264 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-01T19:43:29.264 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-01T19:43:29.264 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-01T19:43:29.264 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -- ceph health --format=json 2023-10-01T19:43:29.525 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-01T19:43:30.208 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:29 smithi087 systemd[1]: Starting Ceph grafana.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:43:30.595 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="The state of unified alerting is still not defined. The decision will be made during as we run the database migrations" logger=settings 2023-10-01T19:43:30.595 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=warn msg="falling back to legacy setting of 'min_interval_seconds'; please use the configuration option in the `unified_alerting` section if Grafana 8 alerts are enabled." logger=settings 2023-10-01T19:43:30.595 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini 2023-10-01T19:43:30.595 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini 2023-10-01T19:43:30.595 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_DATA=/var/lib/grafana" 2023-10-01T19:43:30.596 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_LOGS=/var/log/grafana" 2023-10-01T19:43:30.596 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2023-10-01T19:43:30.596 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2023-10-01T19:43:30.596 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana 2023-10-01T19:43:30.596 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana 2023-10-01T19:43:30.596 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana 2023-10-01T19:43:30.596 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins 2023-10-01T19:43:30.596 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning 2023-10-01T19:43:30.596 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="App mode production" logger=settings 2023-10-01T19:43:30.597 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2023-10-01T19:43:30.597 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=warn msg="SQLite database file has broader permissions than it should" logger=sqlstore path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2023-10-01T19:43:30.597 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Starting DB migrations" logger=migrator 2023-10-01T19:43:30.597 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create migration_log table" 2023-10-01T19:43:30.597 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create user table" 2023-10-01T19:43:30.597 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.login" 2023-10-01T19:43:30.597 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.email" 2023-10-01T19:43:30.597 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_login - v1" 2023-10-01T19:43:30.597 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_email - v1" 2023-10-01T19:43:30.597 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table user to user_v1 - v1" 2023-10-01T19:43:30.598 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create user table v2" 2023-10-01T19:43:30.598 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_login - v2" 2023-10-01T19:43:30.598 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_email - v2" 2023-10-01T19:43:30.598 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 bash[129924]: 19c2251f4393d9eee1748c03781483d216014b824fd6653b90816b0261ed4119 2023-10-01T19:43:30.598 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2023-10-01T19:43:30.598 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table user_v1" 2023-10-01T19:43:30.598 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column help_flags1 to user table" 2023-10-01T19:43:30.598 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update user table charset" 2023-10-01T19:43:30.598 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add last_seen_at column to user" 2023-10-01T19:43:30.598 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add missing user data" 2023-10-01T19:43:30.598 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 systemd[1]: Started Ceph grafana.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:43:30.599 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_disabled column to user" 2023-10-01T19:43:30.599 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index user.login/user.email" 2023-10-01T19:43:30.599 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_service_account column to user" 2023-10-01T19:43:30.599 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create temp user table v1-7" 2023-10-01T19:43:30.599 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v1-7" 2023-10-01T19:43:30.599 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v1-7" 2023-10-01T19:43:30.599 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v1-7" 2023-10-01T19:43:30.599 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v1-7" 2023-10-01T19:43:30.599 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update temp_user table charset" 2023-10-01T19:43:30.599 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_email - v1" 2023-10-01T19:43:30.600 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_org_id - v1" 2023-10-01T19:43:30.600 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_code - v1" 2023-10-01T19:43:30.600 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_status - v1" 2023-10-01T19:43:30.600 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2023-10-01T19:43:30.600 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create temp_user v2" 2023-10-01T19:43:30.600 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v2" 2023-10-01T19:43:30.600 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v2" 2023-10-01T19:43:30.600 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v2" 2023-10-01T19:43:30.600 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v2" 2023-10-01T19:43:30.600 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="copy temp_user v1 to v2" 2023-10-01T19:43:30.600 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop temp_user_tmp_qwerty" 2023-10-01T19:43:30.601 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Set created for temp users that will otherwise prematurely expire" 2023-10-01T19:43:30.601 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create star table" 2023-10-01T19:43:30.601 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index star.user_id_dashboard_id" 2023-10-01T19:43:30.601 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create org table v1" 2023-10-01T19:43:30.601 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_name - v1" 2023-10-01T19:43:30.601 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create org_user table v1" 2023-10-01T19:43:30.601 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_org_id - v1" 2023-10-01T19:43:30.601 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_user_org_id_user_id - v1" 2023-10-01T19:43:30.601 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_user_id - v1" 2023-10-01T19:43:30.601 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update org table charset" 2023-10-01T19:43:30.602 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update org_user table charset" 2023-10-01T19:43:30.602 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate all Read Only Viewers to Viewers" 2023-10-01T19:43:30.602 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard table" 2023-10-01T19:43:30.602 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard.account_id" 2023-10-01T19:43:30.602 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_account_id_slug" 2023-10-01T19:43:30.602 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_tag table" 2023-10-01T19:43:30.602 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_tag.dasboard_id_term" 2023-10-01T19:43:30.602 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2023-10-01T19:43:30.602 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard to dashboard_v1 - v1" 2023-10-01T19:43:30.603 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard v2" 2023-10-01T19:43:30.603 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_org_id - v2" 2023-10-01T19:43:30.603 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_org_id_slug - v2" 2023-10-01T19:43:30.603 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard v1 to v2" 2023-10-01T19:43:30.603 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_v1" 2023-10-01T19:43:30.603 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard.data to mediumtext v1" 2023-10-01T19:43:30.603 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column updated_by in dashboard - v2" 2023-10-01T19:43:30.603 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column created_by in dashboard - v2" 2023-10-01T19:43:30.603 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column gnetId in dashboard" 2023-10-01T19:43:30.603 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for gnetId in dashboard" 2023-10-01T19:43:30.604 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_id in dashboard" 2023-10-01T19:43:30.604 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for plugin_id in dashboard" 2023-10-01T19:43:30.604 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_id in dashboard_tag" 2023-10-01T19:43:30.604 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard table charset" 2023-10-01T19:43:30.604 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_tag table charset" 2023-10-01T19:43:30.604 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column folder_id in dashboard" 2023-10-01T19:43:30.604 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column isFolder in dashboard" 2023-10-01T19:43:30.604 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column has_acl in dashboard" 2023-10-01T19:43:30.604 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in dashboard" 2023-10-01T19:43:30.604 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in dashboard" 2023-10-01T19:43:30.605 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index dashboard_org_id_uid" 2023-10-01T19:43:30.605 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_slug" 2023-10-01T19:43:30.605 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard title length" 2023-10-01T19:43:30.605 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index for dashboard_org_id_title_folder_id" 2023-10-01T19:43:30.605 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning" 2023-10-01T19:43:30.605 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2023-10-01T19:43:30.605 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning v2" 2023-10-01T19:43:30.605 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2023-10-01T19:43:30.605 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2023-10-01T19:43:30.605 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard_provisioning v1 to v2" 2023-10-01T19:43:30.605 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop dashboard_provisioning_tmp_qwerty" 2023-10-01T19:43:30.606 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add check_sum column" 2023-10-01T19:43:30.606 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_title" 2023-10-01T19:43:30.606 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="delete tags for deleted dashboards" 2023-10-01T19:43:30.606 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="delete stars for deleted dashboards" 2023-10-01T19:43:30.606 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_is_folder" 2023-10-01T19:43:30.606 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table" 2023-10-01T19:43:30.606 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index data_source.account_id" 2023-10-01T19:43:30.606 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index data_source.account_id_name" 2023-10-01T19:43:30.606 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_data_source_account_id - v1" 2023-10-01T19:43:30.607 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_data_source_account_id_name - v1" 2023-10-01T19:43:30.607 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table data_source to data_source_v1 - v1" 2023-10-01T19:43:30.607 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table v2" 2023-10-01T19:43:30.607 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_data_source_org_id - v2" 2023-10-01T19:43:30.607 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_data_source_org_id_name - v2" 2023-10-01T19:43:30.607 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2023-10-01T19:43:30.607 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table data_source_v1 #2" 2023-10-01T19:43:30.607 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column with_credentials" 2023-10-01T19:43:30.607 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add secure json data column" 2023-10-01T19:43:30.608 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update data_source table charset" 2023-10-01T19:43:30.608 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update initial version to 1" 2023-10-01T19:43:30.608 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add read_only data column" 2023-10-01T19:43:30.608 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate logging ds to loki ds" 2023-10-01T19:43:30.608 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update json_data with nulls" 2023-10-01T19:43:30.608 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add uid column" 2023-10-01T19:43:30.608 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid value" 2023-10-01T19:43:30.608 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index datasource_org_id_uid" 2023-10-01T19:43:30.608 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index datasource_org_id_is_default" 2023-10-01T19:43:30.608 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table" 2023-10-01T19:43:30.608 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id" 2023-10-01T19:43:30.609 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.key" 2023-10-01T19:43:30.609 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id_name" 2023-10-01T19:43:30.609 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_api_key_account_id - v1" 2023-10-01T19:43:30.609 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_key - v1" 2023-10-01T19:43:30.609 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_account_id_name - v1" 2023-10-01T19:43:30.609 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table api_key to api_key_v1 - v1" 2023-10-01T19:43:30.609 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table v2" 2023-10-01T19:43:30.609 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_api_key_org_id - v2" 2023-10-01T19:43:30.609 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_key - v2" 2023-10-01T19:43:30.609 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_org_id_name - v2" 2023-10-01T19:43:30.610 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="copy api_key v1 to v2" 2023-10-01T19:43:30.610 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table api_key_v1" 2023-10-01T19:43:30.610 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update api_key table charset" 2023-10-01T19:43:30.610 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add expires to api_key table" 2023-10-01T19:43:30.610 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add service account foreign key" 2023-10-01T19:43:30.610 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v4" 2023-10-01T19:43:30.610 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_snapshot_v4 #1" 2023-10-01T19:43:30.610 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v5 #2" 2023-10-01T19:43:30.610 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_key - v5" 2023-10-01T19:43:30.610 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_delete_key - v5" 2023-10-01T19:43:30.611 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_snapshot_user_id - v5" 2023-10-01T19:43:30.611 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_snapshot to mediumtext v2" 2023-10-01T19:43:30.611 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_snapshot table charset" 2023-10-01T19:43:30.611 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external_delete_url to dashboard_snapshots table" 2023-10-01T19:43:30.611 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add encrypted dashboard json column" 2023-10-01T19:43:30.611 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Change dashboard_encrypted column to MEDIUMBLOB" 2023-10-01T19:43:30.611 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create quota table v1" 2023-10-01T19:43:30.611 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_quota_org_id_user_id_target - v1" 2023-10-01T19:43:30.611 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update quota table charset" 2023-10-01T19:43:30.611 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create plugin_setting table" 2023-10-01T19:43:30.612 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2023-10-01T19:43:30.612 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_version to plugin_settings" 2023-10-01T19:43:30.612 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update plugin_setting table charset" 2023-10-01T19:43:30.612 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create session table" 2023-10-01T19:43:30.612 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist table" 2023-10-01T19:43:30.612 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist_item table" 2023-10-01T19:43:30.612 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist table v2" 2023-10-01T19:43:30.612 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist item table v2" 2023-10-01T19:43:30.612 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist table charset" 2023-10-01T19:43:30.612 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist_item table charset" 2023-10-01T19:43:30.612 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v2" 2023-10-01T19:43:30.613 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v3" 2023-10-01T19:43:30.613 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create preferences table v3" 2023-10-01T19:43:30.613 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update preferences table charset" 2023-10-01T19:43:30.613 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column team_id in preferences" 2023-10-01T19:43:30.613 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update team_id column values in preferences" 2023-10-01T19:43:30.613 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column week_start in preferences" 2023-10-01T19:43:30.613 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create alert table v1" 2023-10-01T19:43:30.613 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert org_id & id " 2023-10-01T19:43:30.613 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert state" 2023-10-01T19:43:30.613 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert dashboard_id" 2023-10-01T19:43:30.614 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v1" 2023-10-01T19:43:30.614 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id" 2023-10-01T19:43:30.614 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2023-10-01T19:43:30.614 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2023-10-01T19:43:30.614 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v2" 2023-10-01T19:43:30.614 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2023-10-01T19:43:30.614 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="copy alert_rule_tag v1 to v2" 2023-10-01T19:43:30.614 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop table alert_rule_tag_v1" 2023-10-01T19:43:30.614 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification table v1" 2023-10-01T19:43:30.614 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column is_default" 2023-10-01T19:43:30.615 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column frequency" 2023-10-01T19:43:30.615 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column send_reminder" 2023-10-01T19:43:30.615 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column disable_resolve_message" 2023-10-01T19:43:30.615 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification org_id & name" 2023-10-01T19:43:30.615 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert table charset" 2023-10-01T19:43:30.615 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert_notification table charset" 2023-10-01T19:43:30.615 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create notification_journal table v1" 2023-10-01T19:43:30.615 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index notification_journal org_id & alert_id & notifier_id" 2023-10-01T19:43:30.615 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_notification_journal" 2023-10-01T19:43:30.615 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification_state table v1" 2023-10-01T19:43:30.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[108286]: from='client.24514 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-01T19:43:30.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[108286]: pgmap v31: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:30.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:30.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:30.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:30.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:30.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:30.617 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[114331]: from='client.24514 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-01T19:43:30.617 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[114331]: pgmap v31: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:30.617 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:30.617 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:30.617 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:30.617 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:30.617 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:30.778 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:43:30.778 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-01T19:43:30.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-mon[109386]: from='client.24514 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-01T19:43:30.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-mon[109386]: pgmap v31: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:30.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:30.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:30.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:30.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:30.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:30.852 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id" 2023-10-01T19:43:30.852 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add for to alert table" 2023-10-01T19:43:30.852 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in alert_notification" 2023-10-01T19:43:30.852 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in alert_notification" 2023-10-01T19:43:30.852 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_notification_org_id_uid" 2023-10-01T19:43:30.852 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_name" 2023-10-01T19:43:30.852 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column secure_settings in alert_notification" 2023-10-01T19:43:30.853 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert.settings to mediumtext" 2023-10-01T19:43:30.853 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_notification_state_alert_id" 2023-10-01T19:43:30.853 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_rule_tag_alert_id" 2023-10-01T19:43:30.853 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old annotation table v4" 2023-10-01T19:43:30.853 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create annotation table v5" 2023-10-01T19:43:30.853 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 0 v3" 2023-10-01T19:43:30.853 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 1 v3" 2023-10-01T19:43:30.853 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 2 v3" 2023-10-01T19:43:30.853 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 3 v3" 2023-10-01T19:43:30.853 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 4 v3" 2023-10-01T19:43:30.854 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update annotation table charset" 2023-10-01T19:43:30.854 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column region_id to annotation table" 2023-10-01T19:43:30.854 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Drop category_id index" 2023-10-01T19:43:30.854 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column tags to annotation table" 2023-10-01T19:43:30.854 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v2" 2023-10-01T19:43:30.854 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id" 2023-10-01T19:43:30.854 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2023-10-01T19:43:30.854 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table annotation_tag to annotation_tag_v2 - v2" 2023-10-01T19:43:30.854 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v3" 2023-10-01T19:43:30.854 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2023-10-01T19:43:30.854 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="copy annotation_tag v2 to v3" 2023-10-01T19:43:30.854 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop table annotation_tag_v2" 2023-10-01T19:43:30.855 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert annotations and set TEXT to empty" 2023-10-01T19:43:30.855 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add created time to annotation table" 2023-10-01T19:43:30.855 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add updated time to annotation table" 2023-10-01T19:43:30.855 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for created in annotation table" 2023-10-01T19:43:30.855 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for updated in annotation table" 2023-10-01T19:43:30.855 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Convert existing annotations from seconds to milliseconds" 2023-10-01T19:43:30.855 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add epoch_end column" 2023-10-01T19:43:30.855 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for epoch_end" 2023-10-01T19:43:30.855 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Make epoch_end the same as epoch" 2023-10-01T19:43:30.855 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Move region to single row" 2023-10-01T19:43:30.855 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch from annotation table" 2023-10-01T19:43:30.855 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2023-10-01T19:43:30.856 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2023-10-01T19:43:30.856 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table" 2023-10-01T19:43:30.856 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table" 2023-10-01T19:43:30.856 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for alert_id on annotation table" 2023-10-01T19:43:30.856 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create test_data table" 2023-10-01T19:43:30.856 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_version table v1" 2023-10-01T19:43:30.856 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_version.dashboard_id" 2023-10-01T19:43:30.856 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2023-10-01T19:43:30.856 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Set dashboard version to 1 where 0" 2023-10-01T19:43:30.856 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="save existing dashboard data in dashboard_version table v1" 2023-10-01T19:43:30.856 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_version.data to mediumtext v1" 2023-10-01T19:43:30.856 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create team table" 2023-10-01T19:43:30.857 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index team.org_id" 2023-10-01T19:43:30.857 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_org_id_name" 2023-10-01T19:43:30.857 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create team member table" 2023-10-01T19:43:30.857 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.org_id" 2023-10-01T19:43:30.857 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_member_org_id_team_id_user_id" 2023-10-01T19:43:30.857 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.team_id" 2023-10-01T19:43:30.857 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column email to team table" 2023-10-01T19:43:30.857 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external to team_member table" 2023-10-01T19:43:30.857 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column permission to team_member table" 2023-10-01T19:43:30.857 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard acl table" 2023-10-01T19:43:30.857 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_dashboard_id" 2023-10-01T19:43:30.857 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id" 2023-10-01T19:43:30.858 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id" 2023-10-01T19:43:30.858 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_user_id" 2023-10-01T19:43:30.858 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_team_id" 2023-10-01T19:43:30.858 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_org_id_role" 2023-10-01T19:43:30.858 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_permission" 2023-10-01T19:43:30.858 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="save default acl rules in dashboard_acl table" 2023-10-01T19:43:30.858 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="delete acl rules for deleted dashboards and folders" 2023-10-01T19:43:30.858 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create tag table" 2023-10-01T19:43:30.858 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index tag.key_value" 2023-10-01T19:43:30.858 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create login attempt table" 2023-10-01T19:43:30.858 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index login_attempt.username" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_login_attempt_username - v1" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create login_attempt v2" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_login_attempt_username - v2" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="copy login_attempt v1 to v2" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop login_attempt_tmp_qwerty" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth table" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="alter user_auth.auth_id to length 190" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth access token to user_auth" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth refresh token to user_auth" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth token type to user_auth" 2023-10-01T19:43:30.859 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth expiry to user_auth" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index to user_id column in user_auth" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create server_lock table" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index server_lock.operation_uid" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth token table" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.auth_token" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.prev_auth_token" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_auth_token.user_id" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add revoked_at to the user auth token" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create cache_data table" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index cache_data.cache_key" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create short_url table v1" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index short_url.org_id-uid" 2023-10-01T19:43:30.860 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition table" 2023-10-01T19:43:30.861 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition table" 2023-10-01T19:43:30.861 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and title columns" 2023-10-01T19:43:30.861 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and uid columns" 2023-10-01T19:43:30.861 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition table data column to mediumtext in mysql" 2023-10-01T19:43:30.861 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and title columns" 2023-10-01T19:43:30.861 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and uid columns" 2023-10-01T19:43:30.861 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and title columns" 2023-10-01T19:43:30.861 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and uid columns" 2023-10-01T19:43:30.861 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column paused in alert_definition" 2023-10-01T19:43:30.861 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition table" 2023-10-01T19:43:30.861 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition_version table" 2023-10-01T19:43:30.861 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition_version table" 2023-10-01T19:43:30.862 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_id and version columns" 2023-10-01T19:43:30.862 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_uid and version columns" 2023-10-01T19:43:30.862 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition_version table data column to mediumtext in mysql" 2023-10-01T19:43:30.863 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition_version table" 2023-10-01T19:43:30.863 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_instance table" 2023-10-01T19:43:30.863 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2023-10-01T19:43:30.863 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, current_state columns" 2023-10-01T19:43:30.863 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add column current_state_end to alert_instance" 2023-10-01T19:43:30.863 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, def_uid, current_state on alert_instance" 2023-10-01T19:43:30.863 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, current_state on alert_instance" 2023-10-01T19:43:30.863 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_org_id to rule_org_id in alert_instance" 2023-10-01T19:43:30.863 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_uid to rule_uid in alert_instance" 2023-10-01T19:43:30.863 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, rule_uid, current_state on alert_instance" 2023-10-01T19:43:30.863 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, current_state on alert_instance" 2023-10-01T19:43:30.864 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule table" 2023-10-01T19:43:30.864 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and title columns" 2023-10-01T19:43:30.864 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and uid columns" 2023-10-01T19:43:30.864 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2023-10-01T19:43:30.864 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule table data column to mediumtext in mysql" 2023-10-01T19:43:30.864 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule" 2023-10-01T19:43:30.864 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule" 2023-10-01T19:43:30.864 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule" 2023-10-01T19:43:30.864 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="remove unique index from alert_rule on org_id, title columns" 2023-10-01T19:43:30.864 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespase_uid and title columns" 2023-10-01T19:43:30.864 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add dashboard_uid column to alert_rule" 2023-10-01T19:43:30.864 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add panel_id column to alert_rule" 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule_version table" 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule_version table data column to mediumtext in mysql" 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule_version" 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule_version" 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule_version" 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id=create_alert_configuration_table 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column default in alert_configuration" 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add column org_id in alert_configuration" 2023-10-01T19:43:30.865 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_configuration table on org_id column" 2023-10-01T19:43:30.866 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id=create_ngalert_configuration_table 2023-10-01T19:43:30.866 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index in ngalert_configuration on org_id column" 2023-10-01T19:43:30.866 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="clear migration entry \"remove unified alerting data\"" 2023-10-01T19:43:30.866 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="move dashboard alerts to unified alerting" 2023-10-01T19:43:30.866 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element table v1" 2023-10-01T19:43:30.866 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element org_id-folder_id-name-kind" 2023-10-01T19:43:30.866 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element_connection table v1" 2023-10-01T19:43:30.866 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element_connection element_id-kind-connection_id" 2023-10-01T19:43:30.866 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index library_element org_id_uid" 2023-10-01T19:43:30.866 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="clone move dashboard alerts to unified alerting" 2023-10-01T19:43:30.866 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create data_keys table" 2023-10-01T19:43:30.866 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create kv_store table v1" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index kv_store.org_id-namespace-key" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="update dashboard_uid and panel_id from existing annotations" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create permission table" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index permission.role_id" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_id_action_scope" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create role table" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add column display_name" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add column group_name" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index role.org_id" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_org_id_name" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index role_org_id_uid" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create team role table" 2023-10-01T19:43:30.867 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.org_id" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_role_org_id_team_id_role_id" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.team_id" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create user role table" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.org_id" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_role_org_id_user_id_role_id" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.user_id" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create builtin role table" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.role_id" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.name" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column org_id to builtin_role table" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.org_id" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_org_id_role_id_role" 2023-10-01T19:43:30.868 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index role_org_id_uid" 2023-10-01T19:43:30.869 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role.uid" 2023-10-01T19:43:30.869 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="create seed assignment table" 2023-10-01T19:43:30.869 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_role_name" 2023-10-01T19:43:30.869 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="migrations completed" logger=migrator performed=381 skipped=0 duration=487.23225ms 2023-10-01T19:43:30.869 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Created default organization" logger=sqlstore 2023-10-01T19:43:30.869 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Initialising plugins" logger=plugin.manager 2023-10-01T19:43:30.869 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=input 2023-10-01T19:43:31.133 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=grafana-piechart-panel 2023-10-01T19:43:31.133 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=vonage-status-panel 2023-10-01T19:43:31.133 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="Live Push Gateway initialization" logger=live.push_http 2023-10-01T19:43:31.133 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" logger=provisioning.datasources filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2023-10-01T19:43:31.133 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 uid=P43CA22E17D0F9596 2023-10-01T19:43:31.134 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="warming cache for startup" logger=ngalert 2023-10-01T19:43:31.134 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2023-10-01T19:43:31.134 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:43:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:43:30+0000 lvl=info msg="starting MultiOrg Alertmanager" logger=ngalert.multiorg.alertmanager 2023-10-01T19:43:31.806 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:31 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:43:31.807 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:31 smithi087 ceph-mon[109386]: Reconfiguring alertmanager.a (dependencies changed)... 2023-10-01T19:43:31.807 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:31 smithi087 ceph-mon[109386]: Reconfiguring daemon alertmanager.a on smithi057 2023-10-01T19:43:31.807 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:31 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2256075609' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-01T19:43:31.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:31 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:43:31.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:31 smithi057 ceph-mon[108286]: Reconfiguring alertmanager.a (dependencies changed)... 2023-10-01T19:43:31.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:31 smithi057 ceph-mon[108286]: Reconfiguring daemon alertmanager.a on smithi057 2023-10-01T19:43:31.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:31 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2256075609' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-01T19:43:31.890 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:31 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:43:31.890 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:31 smithi057 ceph-mon[114331]: Reconfiguring alertmanager.a (dependencies changed)... 2023-10-01T19:43:31.890 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:31 smithi057 ceph-mon[114331]: Reconfiguring daemon alertmanager.a on smithi057 2023-10-01T19:43:31.890 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:31 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2256075609' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-01T19:43:32.205 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-01T19:43:32.205 INFO:tasks.cephadm:Setup complete, yielding 2023-10-01T19:43:32.206 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-01T19:43:32.215 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2023-10-01T19:43:32.215 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'radosgw-admin realm create --rgw-realm=r --default' 2023-10-01T19:43:32.620 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:32 smithi057 systemd[1]: Stopping Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:43:32.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:32 smithi057 ceph-mon[108286]: pgmap v32: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:32.863 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:32 smithi087 ceph-mon[109386]: pgmap v32: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:32.889 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:32 smithi057 bash[138704]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager.a" found: no such container 2023-10-01T19:43:32.890 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:32 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[136328]: level=info ts=2023-10-01T19:43:32.679Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2023-10-01T19:43:32.890 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:32 smithi057 ceph-mon[114331]: pgmap v32: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:33.172 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:32 smithi057 bash[138717]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a 2023-10-01T19:43:33.172 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:32 smithi057 bash[138771]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager.a" found: no such container 2023-10-01T19:43:33.172 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:33 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@alertmanager.a.service: Succeeded. 2023-10-01T19:43:33.172 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:33 smithi057 systemd[1]: Stopped Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:43:33.172 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:33 smithi057 systemd[1]: Starting Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:43:33.857 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:33 smithi087 ceph-mon[109386]: pgmap v33: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:33.906 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:33 smithi057 ceph-mon[108286]: pgmap v33: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:33.906 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:33 smithi057 ceph-mon[114331]: pgmap v33: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:34.421 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=info ts=2023-10-01T19:43:34.017Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2023-10-01T19:43:34.421 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=info ts=2023-10-01T19:43:34.017Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2023-10-01T19:43:34.422 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=info ts=2023-10-01T19:43:34.018Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.57 port=9094 2023-10-01T19:43:34.422 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=info ts=2023-10-01T19:43:34.020Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2023-10-01T19:43:34.422 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:34 smithi057 bash[138872]: 1b7e399a166e8781f26201e8aee51fe0b95facd87bcfeaced7e7a0a402f011af 2023-10-01T19:43:34.422 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:34 smithi057 systemd[1]: Started Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:43:34.422 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=info ts=2023-10-01T19:43:34.067Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2023-10-01T19:43:34.422 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=info ts=2023-10-01T19:43:34.068Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2023-10-01T19:43:34.422 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=info ts=2023-10-01T19:43:34.071Z caller=main.go:518 msg=Listening address=:9093 2023-10-01T19:43:34.422 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=info ts=2023-10-01T19:43:34.071Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2023-10-01T19:43:34.856 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:43:34] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:35.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:43:34] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:35.682 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:35 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.682 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:35 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.682 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:35 smithi087 ceph-mon[109386]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T19:43:35.683 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:35 smithi087 ceph-mon[109386]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T19:43:35.683 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:35 smithi087 ceph-mon[109386]: osdmap e55: 8 total, 8 up, 8 in 2023-10-01T19:43:35.683 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:35 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1361906813' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2023-10-01T19:43:35.683 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:35 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.683 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:35 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.683 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:35 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.687 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T19:43:35.687 INFO:teuthology.orchestra.run.smithi057.stdout: "id": "0069e020-f4c2-4d76-a161-dc7727bb3f9a", 2023-10-01T19:43:35.687 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "r", 2023-10-01T19:43:35.687 INFO:teuthology.orchestra.run.smithi057.stdout: "current_period": "fd340c9b-09e1-42bc-b5cd-097f5bdf7a91", 2023-10-01T19:43:35.687 INFO:teuthology.orchestra.run.smithi057.stdout: "epoch": 1 2023-10-01T19:43:35.687 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T19:43:35.701 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.701 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.701 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[108286]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T19:43:35.701 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[108286]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T19:43:35.701 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[108286]: osdmap e55: 8 total, 8 up, 8 in 2023-10-01T19:43:35.701 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1361906813' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2023-10-01T19:43:35.701 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.701 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.701 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.702 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.702 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.702 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[114331]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T19:43:35.702 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[114331]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T19:43:35.702 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[114331]: osdmap e55: 8 total, 8 up, 8 in 2023-10-01T19:43:35.702 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1361906813' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2023-10-01T19:43:35.702 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.702 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:35.702 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:35 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:36.060 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'radosgw-admin zonegroup create --rgw-zonegroup=default --master --default' 2023-10-01T19:43:36.380 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 systemd[1]: Stopping Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:43:36.380 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 bash[131018]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus.a" found: no such container 2023-10-01T19:43:36.380 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:43:36.376Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2023-10-01T19:43:36.380 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:43:36.377Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2023-10-01T19:43:36.380 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:43:36.377Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2023-10-01T19:43:36.380 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:43:36.377Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2023-10-01T19:43:36.380 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:43:36.377Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2023-10-01T19:43:36.380 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:43:36.377Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2023-10-01T19:43:36.381 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:43:36.377Z caller=main.go:828 level=info msg="Scrape manager stopped" 2023-10-01T19:43:36.381 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:43:36.377Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2023-10-01T19:43:36.381 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:43:36.377Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2023-10-01T19:43:36.381 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:43:36.377Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2023-10-01T19:43:36.381 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:43:36.377Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2023-10-01T19:43:36.381 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[127674]: ts=2023-10-01T19:43:36.377Z caller=main.go:1066 level=info msg="See you next time!" 2023-10-01T19:43:36.416 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:36 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=info ts=2023-10-01T19:43:36.021Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000934691s 2023-10-01T19:43:36.632 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 bash[131030]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a 2023-10-01T19:43:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-mon[109386]: pgmap v35: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1361906813' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2023-10-01T19:43:36.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:36 smithi087 ceph-mon[109386]: osdmap e56: 8 total, 8 up, 8 in 2023-10-01T19:43:36.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:36 smithi057 ceph-mon[108286]: pgmap v35: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:36.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:36 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1361906813' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2023-10-01T19:43:36.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:36 smithi057 ceph-mon[108286]: osdmap e56: 8 total, 8 up, 8 in 2023-10-01T19:43:36.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:36 smithi057 ceph-mon[114331]: pgmap v35: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:43:36.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:36 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1361906813' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2023-10-01T19:43:36.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:36 smithi057 ceph-mon[114331]: osdmap e56: 8 total, 8 up, 8 in 2023-10-01T19:43:37.026 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 bash[131067]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus.a" found: no such container 2023-10-01T19:43:37.026 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@prometheus.a.service: Succeeded. 2023-10-01T19:43:37.027 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 systemd[1]: Stopped Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:43:37.027 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:36 smithi087 systemd[1]: Starting Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:43:37.027 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.026Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2023-10-01T19:43:37.027 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.026Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2023-10-01T19:43:37.027 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.026Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2023-10-01T19:43:37.027 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.026Z caller=main.go:518 level=info host_details="(Linux 4.18.0-514.el8.x86_64 #1 SMP Tue Sep 19 13:19:58 UTC 2023 x86_64 smithi087 (none))" 2023-10-01T19:43:37.027 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.026Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2023-10-01T19:43:37.027 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.026Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2023-10-01T19:43:37.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.027Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2023-10-01T19:43:37.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.028Z caller=main.go:923 level=info msg="Starting TSDB ..." 2023-10-01T19:43:37.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.029Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2023-10-01T19:43:37.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 bash[131132]: e4b29328945a33813644c2cd69d160d310b09117a9a2ffce5c151e32ef2c0243 2023-10-01T19:43:37.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.031Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2023-10-01T19:43:37.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.031Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.776µs 2023-10-01T19:43:37.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.031Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2023-10-01T19:43:37.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 systemd[1]: Started Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:43:37.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.032Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2023-10-01T19:43:37.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.033Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2023-10-01T19:43:37.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.033Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=27.319µs wal_replay_duration=1.595306ms total_replay_duration=1.636803ms 2023-10-01T19:43:37.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.033Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2023-10-01T19:43:37.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.033Z caller=main.go:947 level=info msg="TSDB started" 2023-10-01T19:43:37.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.033Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2023-10-01T19:43:37.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.048Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.415767ms db_storage=716ns remote_storage=1.355µs web_handler=400ns query_engine=697ns scrape=369.229µs scrape_sd=26.358µs notify=18.957µs notify_sd=6.041µs rules=13.775485ms 2023-10-01T19:43:37.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:43:37.048Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2023-10-01T19:43:37.701 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T19:43:37.701 INFO:teuthology.orchestra.run.smithi057.stdout: "id": "efed5cde-5bc1-4454-b7b9-7e7b2f31c940", 2023-10-01T19:43:37.701 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default", 2023-10-01T19:43:37.701 INFO:teuthology.orchestra.run.smithi057.stdout: "api_name": "default", 2023-10-01T19:43:37.701 INFO:teuthology.orchestra.run.smithi057.stdout: "is_master": "true", 2023-10-01T19:43:37.701 INFO:teuthology.orchestra.run.smithi057.stdout: "endpoints": [], 2023-10-01T19:43:37.701 INFO:teuthology.orchestra.run.smithi057.stdout: "hostnames": [], 2023-10-01T19:43:37.702 INFO:teuthology.orchestra.run.smithi057.stdout: "hostnames_s3website": [], 2023-10-01T19:43:37.702 INFO:teuthology.orchestra.run.smithi057.stdout: "master_zone": "", 2023-10-01T19:43:37.702 INFO:teuthology.orchestra.run.smithi057.stdout: "zones": [], 2023-10-01T19:43:37.702 INFO:teuthology.orchestra.run.smithi057.stdout: "placement_targets": [], 2023-10-01T19:43:37.702 INFO:teuthology.orchestra.run.smithi057.stdout: "default_placement": "", 2023-10-01T19:43:37.702 INFO:teuthology.orchestra.run.smithi057.stdout: "realm_id": "0069e020-f4c2-4d76-a161-dc7727bb3f9a", 2023-10-01T19:43:37.702 INFO:teuthology.orchestra.run.smithi057.stdout: "sync_policy": { 2023-10-01T19:43:37.702 INFO:teuthology.orchestra.run.smithi057.stdout: "groups": [] 2023-10-01T19:43:37.702 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:43:37.702 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T19:43:37.841 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: osdmap e57: 8 total, 8 up, 8 in 2023-10-01T19:43:37.841 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: pgmap v38: 33 pgs: 33 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2023-10-01T19:43:37.841 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:37.841 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T19:43:37.841 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T19:43:37.841 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.57:9093"}]: dispatch 2023-10-01T19:43:37.841 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.57:9093"}]: dispatch 2023-10-01T19:43:37.841 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:37.841 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:43:37.841 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:43:37.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T19:43:37.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T19:43:37.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.87:3000"}]: dispatch 2023-10-01T19:43:37.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.87:3000"}]: dispatch 2023-10-01T19:43:37.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:37.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:37.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:37.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:37.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:37.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:37.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:37.843 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:37.843 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:37.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: osdmap e57: 8 total, 8 up, 8 in 2023-10-01T19:43:37.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: pgmap v38: 33 pgs: 33 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2023-10-01T19:43:37.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:37.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T19:43:37.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T19:43:37.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.57:9093"}]: dispatch 2023-10-01T19:43:37.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.57:9093"}]: dispatch 2023-10-01T19:43:37.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.87:3000"}]: dispatch 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.87:3000"}]: dispatch 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:37.844 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:37.845 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:37.845 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:37 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:38.062 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=z --master --default' 2023-10-01T19:43:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: osdmap e57: 8 total, 8 up, 8 in 2023-10-01T19:43:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: pgmap v38: 33 pgs: 33 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2023-10-01T19:43:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T19:43:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T19:43:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.57:9093"}]: dispatch 2023-10-01T19:43:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.57:9093"}]: dispatch 2023-10-01T19:43:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:43:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:43:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T19:43:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T19:43:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.87:3000"}]: dispatch 2023-10-01T19:43:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.87:3000"}]: dispatch 2023-10-01T19:43:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:38.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:38.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:38.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:37 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:39.353 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T19:43:39.354 INFO:teuthology.orchestra.run.smithi057.stdout: "id": "54fc0210-91d0-4ce1-ad83-8771819fbddc", 2023-10-01T19:43:39.354 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "z", 2023-10-01T19:43:39.354 INFO:teuthology.orchestra.run.smithi057.stdout: "domain_root": "z.rgw.meta:root", 2023-10-01T19:43:39.354 INFO:teuthology.orchestra.run.smithi057.stdout: "control_pool": "z.rgw.control", 2023-10-01T19:43:39.354 INFO:teuthology.orchestra.run.smithi057.stdout: "gc_pool": "z.rgw.log:gc", 2023-10-01T19:43:39.354 INFO:teuthology.orchestra.run.smithi057.stdout: "lc_pool": "z.rgw.log:lc", 2023-10-01T19:43:39.354 INFO:teuthology.orchestra.run.smithi057.stdout: "log_pool": "z.rgw.log", 2023-10-01T19:43:39.354 INFO:teuthology.orchestra.run.smithi057.stdout: "intent_log_pool": "z.rgw.log:intent", 2023-10-01T19:43:39.354 INFO:teuthology.orchestra.run.smithi057.stdout: "usage_log_pool": "z.rgw.log:usage", 2023-10-01T19:43:39.354 INFO:teuthology.orchestra.run.smithi057.stdout: "roles_pool": "z.rgw.meta:roles", 2023-10-01T19:43:39.354 INFO:teuthology.orchestra.run.smithi057.stdout: "reshard_pool": "z.rgw.log:reshard", 2023-10-01T19:43:39.354 INFO:teuthology.orchestra.run.smithi057.stdout: "user_keys_pool": "z.rgw.meta:users.keys", 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: "user_email_pool": "z.rgw.meta:users.email", 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: "user_swift_pool": "z.rgw.meta:users.swift", 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: "user_uid_pool": "z.rgw.meta:users.uid", 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: "otp_pool": "z.rgw.otp", 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: "system_key": { 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: "access_key": "", 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: "secret_key": "" 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: "placement_pools": [ 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "default-placement", 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: "val": { 2023-10-01T19:43:39.355 INFO:teuthology.orchestra.run.smithi057.stdout: "index_pool": "z.rgw.buckets.index", 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_classes": { 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout: "STANDARD": { 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout: "data_pool": "z.rgw.buckets.data" 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout: "data_extra_pool": "z.rgw.buckets.non-ec", 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout: "index_type": 0 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout: "realm_id": "0069e020-f4c2-4d76-a161-dc7727bb3f9a", 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout: "notif_pool": "z.rgw.log:notif" 2023-10-01T19:43:39.356 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T19:43:40.020 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'radosgw-admin period update --rgw-realm=r --commit' 2023-10-01T19:43:40.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:40 smithi087 ceph-mon[109386]: pgmap v39: 33 pgs: 33 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2023-10-01T19:43:40.385 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:40 smithi057 ceph-mon[108286]: pgmap v39: 33 pgs: 33 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2023-10-01T19:43:40.386 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:40 smithi057 ceph-mon[114331]: pgmap v39: 33 pgs: 33 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2023-10-01T19:43:42.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:42 smithi087 ceph-mon[109386]: pgmap v40: 33 pgs: 33 active+clean; 579 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-10-01T19:43:42.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:42 smithi057 ceph-mon[108286]: pgmap v40: 33 pgs: 33 active+clean; 579 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-10-01T19:43:42.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:42 smithi057 ceph-mon[114331]: pgmap v40: 33 pgs: 33 active+clean; 579 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-10-01T19:43:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:43 smithi057 ceph-mon[108286]: osdmap e58: 8 total, 8 up, 8 in 2023-10-01T19:43:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:43 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1749497237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2023-10-01T19:43:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:43 smithi057 ceph-mon[114331]: osdmap e58: 8 total, 8 up, 8 in 2023-10-01T19:43:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:43 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1749497237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2023-10-01T19:43:43.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:43 smithi087 ceph-mon[109386]: osdmap e58: 8 total, 8 up, 8 in 2023-10-01T19:43:43.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:43 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1749497237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2023-10-01T19:43:43.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:43:43] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:44.421 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:43:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=info ts=2023-10-01T19:43:44.023Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.003289337s 2023-10-01T19:43:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:44 smithi057 ceph-mon[108286]: pgmap v42: 65 pgs: 32 unknown, 33 active+clean; 579 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 2.2 KiB/s wr, 3 op/s 2023-10-01T19:43:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:44 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1749497237' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2023-10-01T19:43:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:44 smithi057 ceph-mon[108286]: osdmap e59: 8 total, 8 up, 8 in 2023-10-01T19:43:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:44 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:44 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:44.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:44 smithi057 ceph-mon[114331]: pgmap v42: 65 pgs: 32 unknown, 33 active+clean; 579 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 2.2 KiB/s wr, 3 op/s 2023-10-01T19:43:44.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:44 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1749497237' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2023-10-01T19:43:44.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:44 smithi057 ceph-mon[114331]: osdmap e59: 8 total, 8 up, 8 in 2023-10-01T19:43:44.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:44 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:44.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:44 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:44.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:44 smithi087 ceph-mon[109386]: pgmap v42: 65 pgs: 32 unknown, 33 active+clean; 579 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 2.2 KiB/s wr, 3 op/s 2023-10-01T19:43:44.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:44 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1749497237' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2023-10-01T19:43:44.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:44 smithi087 ceph-mon[109386]: osdmap e59: 8 total, 8 up, 8 in 2023-10-01T19:43:44.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:44 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:44.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:44 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:45.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:45 smithi057 ceph-mon[108286]: osdmap e60: 8 total, 8 up, 8 in 2023-10-01T19:43:45.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:45 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1749497237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2023-10-01T19:43:45.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:43:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:43:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:45 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1749497237' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2023-10-01T19:43:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:45 smithi057 ceph-mon[108286]: osdmap e61: 8 total, 8 up, 8 in 2023-10-01T19:43:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:45 smithi057 ceph-mon[114331]: osdmap e60: 8 total, 8 up, 8 in 2023-10-01T19:43:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:45 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1749497237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2023-10-01T19:43:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:43:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:43:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:45 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1749497237' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2023-10-01T19:43:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:45 smithi057 ceph-mon[114331]: osdmap e61: 8 total, 8 up, 8 in 2023-10-01T19:43:45.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:45 smithi087 ceph-mon[109386]: osdmap e60: 8 total, 8 up, 8 in 2023-10-01T19:43:45.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:45 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1749497237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2023-10-01T19:43:45.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:43:45.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:43:45.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:45 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1749497237' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2023-10-01T19:43:45.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:45 smithi087 ceph-mon[109386]: osdmap e61: 8 total, 8 up, 8 in 2023-10-01T19:43:46.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:46 smithi057 ceph-mon[108286]: pgmap v45: 97 pgs: 64 unknown, 33 active+clean; 579 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2023-10-01T19:43:46.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:43:46] "GET /metrics HTTP/1.1" 200 195172 "" "Prometheus/2.33.4" 2023-10-01T19:43:46.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:46 smithi057 ceph-mon[114331]: pgmap v45: 97 pgs: 64 unknown, 33 active+clean; 579 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2023-10-01T19:43:46.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:46 smithi087 ceph-mon[109386]: pgmap v45: 97 pgs: 64 unknown, 33 active+clean; 579 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2023-10-01T19:43:47.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:47 smithi087 ceph-mon[109386]: osdmap e62: 8 total, 8 up, 8 in 2023-10-01T19:43:47.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:47 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/926281935' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2023-10-01T19:43:47.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:47 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2023-10-01T19:43:47.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:47 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2023-10-01T19:43:47.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:47 smithi087 ceph-mon[109386]: osdmap e63: 8 total, 8 up, 8 in 2023-10-01T19:43:47.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:47 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/926281935' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2023-10-01T19:43:47.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:47 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2023-10-01T19:43:47.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[114331]: osdmap e62: 8 total, 8 up, 8 in 2023-10-01T19:43:47.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/926281935' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2023-10-01T19:43:47.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2023-10-01T19:43:47.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2023-10-01T19:43:47.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[114331]: osdmap e63: 8 total, 8 up, 8 in 2023-10-01T19:43:47.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/926281935' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2023-10-01T19:43:47.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2023-10-01T19:43:47.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[108286]: osdmap e62: 8 total, 8 up, 8 in 2023-10-01T19:43:47.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/926281935' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2023-10-01T19:43:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2023-10-01T19:43:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2023-10-01T19:43:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[108286]: osdmap e63: 8 total, 8 up, 8 in 2023-10-01T19:43:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/926281935' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2023-10-01T19:43:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:47 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2023-10-01T19:43:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:48 smithi087 ceph-mon[109386]: pgmap v48: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-01T19:43:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:48 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2023-10-01T19:43:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:48 smithi087 ceph-mon[109386]: osdmap e64: 8 total, 8 up, 8 in 2023-10-01T19:43:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:48 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/926281935' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2023-10-01T19:43:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:48 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2023-10-01T19:43:48.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:48 smithi057 ceph-mon[108286]: pgmap v48: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-01T19:43:48.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:48 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2023-10-01T19:43:48.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:48 smithi057 ceph-mon[108286]: osdmap e64: 8 total, 8 up, 8 in 2023-10-01T19:43:48.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:48 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/926281935' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2023-10-01T19:43:48.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:48 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2023-10-01T19:43:48.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:48 smithi057 ceph-mon[114331]: pgmap v48: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-01T19:43:48.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:48 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2023-10-01T19:43:48.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:48 smithi057 ceph-mon[114331]: osdmap e64: 8 total, 8 up, 8 in 2023-10-01T19:43:48.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:48 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/926281935' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2023-10-01T19:43:48.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:48 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2023-10-01T19:43:49.252 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T19:43:49.252 INFO:teuthology.orchestra.run.smithi057.stdout: "id": "60887a34-7ed2-4cff-8021-005b80732ffe", 2023-10-01T19:43:49.252 INFO:teuthology.orchestra.run.smithi057.stdout: "epoch": 1, 2023-10-01T19:43:49.252 INFO:teuthology.orchestra.run.smithi057.stdout: "predecessor_uuid": "fd340c9b-09e1-42bc-b5cd-097f5bdf7a91", 2023-10-01T19:43:49.252 INFO:teuthology.orchestra.run.smithi057.stdout: "sync_status": [], 2023-10-01T19:43:49.253 INFO:teuthology.orchestra.run.smithi057.stdout: "period_map": { 2023-10-01T19:43:49.253 INFO:teuthology.orchestra.run.smithi057.stdout: "id": "60887a34-7ed2-4cff-8021-005b80732ffe", 2023-10-01T19:43:49.253 INFO:teuthology.orchestra.run.smithi057.stdout: "zonegroups": [ 2023-10-01T19:43:49.253 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-01T19:43:49.253 INFO:teuthology.orchestra.run.smithi057.stdout: "id": "efed5cde-5bc1-4454-b7b9-7e7b2f31c940", 2023-10-01T19:43:49.253 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default", 2023-10-01T19:43:49.253 INFO:teuthology.orchestra.run.smithi057.stdout: "api_name": "default", 2023-10-01T19:43:49.253 INFO:teuthology.orchestra.run.smithi057.stdout: "is_master": "true", 2023-10-01T19:43:49.253 INFO:teuthology.orchestra.run.smithi057.stdout: "endpoints": [], 2023-10-01T19:43:49.253 INFO:teuthology.orchestra.run.smithi057.stdout: "hostnames": [], 2023-10-01T19:43:49.253 INFO:teuthology.orchestra.run.smithi057.stdout: "hostnames_s3website": [], 2023-10-01T19:43:49.253 INFO:teuthology.orchestra.run.smithi057.stdout: "master_zone": "54fc0210-91d0-4ce1-ad83-8771819fbddc", 2023-10-01T19:43:49.254 INFO:teuthology.orchestra.run.smithi057.stdout: "zones": [ 2023-10-01T19:43:49.254 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-01T19:43:49.254 INFO:teuthology.orchestra.run.smithi057.stdout: "id": "54fc0210-91d0-4ce1-ad83-8771819fbddc", 2023-10-01T19:43:49.254 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "z", 2023-10-01T19:43:49.254 INFO:teuthology.orchestra.run.smithi057.stdout: "endpoints": [], 2023-10-01T19:43:49.254 INFO:teuthology.orchestra.run.smithi057.stdout: "log_meta": "false", 2023-10-01T19:43:49.254 INFO:teuthology.orchestra.run.smithi057.stdout: "log_data": "false", 2023-10-01T19:43:49.254 INFO:teuthology.orchestra.run.smithi057.stdout: "bucket_index_max_shards": 11, 2023-10-01T19:43:49.254 INFO:teuthology.orchestra.run.smithi057.stdout: "read_only": "false", 2023-10-01T19:43:49.254 INFO:teuthology.orchestra.run.smithi057.stdout: "tier_type": "", 2023-10-01T19:43:49.254 INFO:teuthology.orchestra.run.smithi057.stdout: "sync_from_all": "true", 2023-10-01T19:43:49.255 INFO:teuthology.orchestra.run.smithi057.stdout: "sync_from": [], 2023-10-01T19:43:49.255 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_zone": "" 2023-10-01T19:43:49.255 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:43:49.255 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-01T19:43:49.255 INFO:teuthology.orchestra.run.smithi057.stdout: "placement_targets": [ 2023-10-01T19:43:49.255 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-01T19:43:49.255 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default-placement", 2023-10-01T19:43:49.255 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-01T19:43:49.255 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_classes": [ 2023-10-01T19:43:49.255 INFO:teuthology.orchestra.run.smithi057.stdout: "STANDARD" 2023-10-01T19:43:49.255 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-01T19:43:49.255 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:43:49.256 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-01T19:43:49.256 INFO:teuthology.orchestra.run.smithi057.stdout: "default_placement": "default-placement", 2023-10-01T19:43:49.256 INFO:teuthology.orchestra.run.smithi057.stdout: "realm_id": "0069e020-f4c2-4d76-a161-dc7727bb3f9a", 2023-10-01T19:43:49.256 INFO:teuthology.orchestra.run.smithi057.stdout: "sync_policy": { 2023-10-01T19:43:49.256 INFO:teuthology.orchestra.run.smithi057.stdout: "groups": [] 2023-10-01T19:43:49.256 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:43:49.256 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:43:49.256 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-01T19:43:49.256 INFO:teuthology.orchestra.run.smithi057.stdout: "short_zone_ids": [ 2023-10-01T19:43:49.256 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-01T19:43:49.256 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "54fc0210-91d0-4ce1-ad83-8771819fbddc", 2023-10-01T19:43:49.256 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 2670731549 2023-10-01T19:43:49.257 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:43:49.257 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-01T19:43:49.257 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:43:49.257 INFO:teuthology.orchestra.run.smithi057.stdout: "master_zonegroup": "efed5cde-5bc1-4454-b7b9-7e7b2f31c940", 2023-10-01T19:43:49.257 INFO:teuthology.orchestra.run.smithi057.stdout: "master_zone": "54fc0210-91d0-4ce1-ad83-8771819fbddc", 2023-10-01T19:43:49.257 INFO:teuthology.orchestra.run.smithi057.stdout: "period_config": { 2023-10-01T19:43:49.257 INFO:teuthology.orchestra.run.smithi057.stdout: "bucket_quota": { 2023-10-01T19:43:49.257 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": false, 2023-10-01T19:43:49.257 INFO:teuthology.orchestra.run.smithi057.stdout: "check_on_raw": false, 2023-10-01T19:43:49.257 INFO:teuthology.orchestra.run.smithi057.stdout: "max_size": -1, 2023-10-01T19:43:49.257 INFO:teuthology.orchestra.run.smithi057.stdout: "max_size_kb": 0, 2023-10-01T19:43:49.257 INFO:teuthology.orchestra.run.smithi057.stdout: "max_objects": -1 2023-10-01T19:43:49.258 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:43:49.258 INFO:teuthology.orchestra.run.smithi057.stdout: "user_quota": { 2023-10-01T19:43:49.258 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": false, 2023-10-01T19:43:49.258 INFO:teuthology.orchestra.run.smithi057.stdout: "check_on_raw": false, 2023-10-01T19:43:49.258 INFO:teuthology.orchestra.run.smithi057.stdout: "max_size": -1, 2023-10-01T19:43:49.258 INFO:teuthology.orchestra.run.smithi057.stdout: "max_size_kb": 0, 2023-10-01T19:43:49.258 INFO:teuthology.orchestra.run.smithi057.stdout: "max_objects": -1 2023-10-01T19:43:49.258 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:43:49.258 INFO:teuthology.orchestra.run.smithi057.stdout: "user_ratelimit": { 2023-10-01T19:43:49.258 INFO:teuthology.orchestra.run.smithi057.stdout: "max_read_ops": 0, 2023-10-01T19:43:49.258 INFO:teuthology.orchestra.run.smithi057.stdout: "max_write_ops": 0, 2023-10-01T19:43:49.258 INFO:teuthology.orchestra.run.smithi057.stdout: "max_read_bytes": 0, 2023-10-01T19:43:49.259 INFO:teuthology.orchestra.run.smithi057.stdout: "max_write_bytes": 0, 2023-10-01T19:43:49.259 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": false 2023-10-01T19:43:49.259 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:43:49.259 INFO:teuthology.orchestra.run.smithi057.stdout: "bucket_ratelimit": { 2023-10-01T19:43:49.259 INFO:teuthology.orchestra.run.smithi057.stdout: "max_read_ops": 0, 2023-10-01T19:43:49.259 INFO:teuthology.orchestra.run.smithi057.stdout: "max_write_ops": 0, 2023-10-01T19:43:49.259 INFO:teuthology.orchestra.run.smithi057.stdout: "max_read_bytes": 0, 2023-10-01T19:43:49.259 INFO:teuthology.orchestra.run.smithi057.stdout: "max_write_bytes": 0, 2023-10-01T19:43:49.259 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": false 2023-10-01T19:43:49.259 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:43:49.259 INFO:teuthology.orchestra.run.smithi057.stdout: "anonymous_ratelimit": { 2023-10-01T19:43:49.259 INFO:teuthology.orchestra.run.smithi057.stdout: "max_read_ops": 0, 2023-10-01T19:43:49.260 INFO:teuthology.orchestra.run.smithi057.stdout: "max_write_ops": 0, 2023-10-01T19:43:49.260 INFO:teuthology.orchestra.run.smithi057.stdout: "max_read_bytes": 0, 2023-10-01T19:43:49.260 INFO:teuthology.orchestra.run.smithi057.stdout: "max_write_bytes": 0, 2023-10-01T19:43:49.260 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": false 2023-10-01T19:43:49.260 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:43:49.260 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:43:49.260 INFO:teuthology.orchestra.run.smithi057.stdout: "realm_id": "0069e020-f4c2-4d76-a161-dc7727bb3f9a", 2023-10-01T19:43:49.260 INFO:teuthology.orchestra.run.smithi057.stdout: "realm_name": "r", 2023-10-01T19:43:49.260 INFO:teuthology.orchestra.run.smithi057.stdout: "realm_epoch": 2 2023-10-01T19:43:49.260 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T19:43:49.651 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch apply rgw foo --realm r --zone z --placement=2 --port=8000' 2023-10-01T19:43:50.180 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:43:49 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:43:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:43:50.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:50 smithi057 ceph-mon[108286]: pgmap v51: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-01T19:43:50.365 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:50 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2023-10-01T19:43:50.365 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:50 smithi057 ceph-mon[108286]: osdmap e65: 8 total, 8 up, 8 in 2023-10-01T19:43:50.365 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:50 smithi057 ceph-mon[114331]: pgmap v51: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-01T19:43:50.365 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:50 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2023-10-01T19:43:50.365 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:50 smithi057 ceph-mon[114331]: osdmap e65: 8 total, 8 up, 8 in 2023-10-01T19:43:50.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:50 smithi087 ceph-mon[109386]: pgmap v51: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-01T19:43:50.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:50 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2023-10-01T19:43:50.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:50 smithi087 ceph-mon[109386]: osdmap e65: 8 total, 8 up, 8 in 2023-10-01T19:43:51.129 INFO:teuthology.orchestra.run.smithi057.stdout:Scheduled rgw.foo update... 2023-10-01T19:43:51.516 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph osd pool create foo' 2023-10-01T19:43:52.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: pgmap v53: 129 pgs: 129 active+clean; 581 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2023-10-01T19:43:52.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:43:52.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: Saving service rgw.foo spec with placement count:2 2023-10-01T19:43:52.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: Saving service rgw.foo spec with placement count:2 2023-10-01T19:43:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi057.dbidoa", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T19:43:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi057.dbidoa", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2023-10-01T19:43:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:52.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[108286]: Deploying daemon rgw.foo.smithi057.dbidoa on smithi057 2023-10-01T19:43:52.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: pgmap v53: 129 pgs: 129 active+clean; 581 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2023-10-01T19:43:52.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:43:52.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: Saving service rgw.foo spec with placement count:2 2023-10-01T19:43:52.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:52.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:52.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:52.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: Saving service rgw.foo spec with placement count:2 2023-10-01T19:43:52.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi057.dbidoa", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T19:43:52.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi057.dbidoa", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2023-10-01T19:43:52.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:52.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:52 smithi057 ceph-mon[114331]: Deploying daemon rgw.foo.smithi057.dbidoa on smithi057 2023-10-01T19:43:52.441 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: pgmap v53: 129 pgs: 129 active+clean; 581 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2023-10-01T19:43:52.441 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:43:52.442 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: Saving service rgw.foo spec with placement count:2 2023-10-01T19:43:52.442 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.442 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:52.442 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:52.442 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:52.442 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.442 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.442 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.442 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: Saving service rgw.foo spec with placement count:2 2023-10-01T19:43:52.442 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.443 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi057.dbidoa", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T19:43:52.443 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi057.dbidoa", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2023-10-01T19:43:52.443 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:52.443 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:52.443 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:52 smithi087 ceph-mon[109386]: Deploying daemon rgw.foo.smithi057.dbidoa on smithi057 2023-10-01T19:43:54.164 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'foo' created 2023-10-01T19:43:54.191 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[108286]: pgmap v54: 129 pgs: 129 active+clean; 581 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2023-10-01T19:43:54.191 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/135898440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2023-10-01T19:43:54.191 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2023-10-01T19:43:54.191 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:54.192 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi087.rxeurp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T19:43:54.192 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi087.rxeurp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2023-10-01T19:43:54.192 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:54.192 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:54.192 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[114331]: pgmap v54: 129 pgs: 129 active+clean; 581 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2023-10-01T19:43:54.192 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/135898440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2023-10-01T19:43:54.192 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2023-10-01T19:43:54.192 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:54.192 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi087.rxeurp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T19:43:54.192 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi087.rxeurp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2023-10-01T19:43:54.193 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:54.193 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:54 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:54.430 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:54 smithi087 ceph-mon[109386]: pgmap v54: 129 pgs: 129 active+clean; 581 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2023-10-01T19:43:54.431 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:54 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/135898440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2023-10-01T19:43:54.431 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:54 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2023-10-01T19:43:54.431 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:54.431 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi087.rxeurp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T19:43:54.431 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi087.rxeurp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2023-10-01T19:43:54.431 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:54.431 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:54 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:54.884 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'rbd pool init foo' 2023-10-01T19:43:55.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:55 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:43:54] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:55.447 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:55 smithi057 ceph-mon[114331]: Deploying daemon rgw.foo.smithi087.rxeurp on smithi087 2023-10-01T19:43:55.447 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:55 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2023-10-01T19:43:55.448 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:55 smithi057 ceph-mon[114331]: osdmap e66: 8 total, 8 up, 8 in 2023-10-01T19:43:55.448 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:55 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:55.448 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:55 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:55.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:55 smithi057 ceph-mon[108286]: Deploying daemon rgw.foo.smithi087.rxeurp on smithi087 2023-10-01T19:43:55.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:55 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2023-10-01T19:43:55.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:55 smithi057 ceph-mon[108286]: osdmap e66: 8 total, 8 up, 8 in 2023-10-01T19:43:55.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:55 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:55.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:55 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:55.545 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:55 smithi087 ceph-mon[109386]: Deploying daemon rgw.foo.smithi087.rxeurp on smithi087 2023-10-01T19:43:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:55 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2023-10-01T19:43:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:55 smithi087 ceph-mon[109386]: osdmap e66: 8 total, 8 up, 8 in 2023-10-01T19:43:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:55 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:55 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:55.747 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:55 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:43:55] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:56.331 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:43:56] "GET /metrics HTTP/1.1" 200 195172 "" "Prometheus/2.33.4" 2023-10-01T19:43:56.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[108286]: pgmap v56: 161 pgs: 32 unknown, 129 active+clean; 581 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2023-10-01T19:43:56.332 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[108286]: osdmap e67: 8 total, 8 up, 8 in 2023-10-01T19:43:56.332 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:56.332 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:56.332 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:56.332 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:56.332 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:56.333 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:56.333 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[114331]: pgmap v56: 161 pgs: 32 unknown, 129 active+clean; 581 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2023-10-01T19:43:56.333 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[114331]: osdmap e67: 8 total, 8 up, 8 in 2023-10-01T19:43:56.333 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:56.333 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:56.333 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:56.334 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:56.334 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:56.334 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:56 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:56 smithi087 ceph-mon[109386]: pgmap v56: 161 pgs: 32 unknown, 129 active+clean; 581 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2023-10-01T19:43:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:56 smithi087 ceph-mon[109386]: osdmap e67: 8 total, 8 up, 8 in 2023-10-01T19:43:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:56 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:56 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:56 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:56 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:56 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:56 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:57.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: Checking dashboard <-> RGW credentials 2023-10-01T19:43:57.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: osdmap e68: 8 total, 8 up, 8 in 2023-10-01T19:43:57.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:57.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:57.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:57.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:43:57.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:57.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:57.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:57.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: Checking dashboard <-> RGW credentials 2023-10-01T19:43:57.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1205916807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2023-10-01T19:43:57.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2023-10-01T19:43:57.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:57.360 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: Checking dashboard <-> RGW credentials 2023-10-01T19:43:57.360 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: osdmap e68: 8 total, 8 up, 8 in 2023-10-01T19:43:57.360 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:57.360 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:57.360 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:57.360 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:43:57.360 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:57.360 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:57.360 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:57.360 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: Checking dashboard <-> RGW credentials 2023-10-01T19:43:57.360 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1205916807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2023-10-01T19:43:57.361 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2023-10-01T19:43:57.361 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:57 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:57.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: Checking dashboard <-> RGW credentials 2023-10-01T19:43:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: osdmap e68: 8 total, 8 up, 8 in 2023-10-01T19:43:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:43:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:43:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:43:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:43:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: Checking dashboard <-> RGW credentials 2023-10-01T19:43:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1205916807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2023-10-01T19:43:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2023-10-01T19:43:57.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:57 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:58.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:58 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:43:58.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:58 smithi057 ceph-mon[114331]: pgmap v59: 161 pgs: 161 active+clean; 584 KiB data, 53 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 5.5 KiB/s wr, 186 op/s 2023-10-01T19:43:58.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:58 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2023-10-01T19:43:58.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:58 smithi057 ceph-mon[114331]: osdmap e69: 8 total, 8 up, 8 in 2023-10-01T19:43:58.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:58 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:43:58.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:58 smithi057 ceph-mon[108286]: pgmap v59: 161 pgs: 161 active+clean; 584 KiB data, 53 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 5.5 KiB/s wr, 186 op/s 2023-10-01T19:43:58.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:58 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2023-10-01T19:43:58.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:58 smithi057 ceph-mon[108286]: osdmap e69: 8 total, 8 up, 8 in 2023-10-01T19:43:58.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:58 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:43:58.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:58 smithi087 ceph-mon[109386]: pgmap v59: 161 pgs: 161 active+clean; 584 KiB data, 53 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 5.5 KiB/s wr, 186 op/s 2023-10-01T19:43:58.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:58 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2023-10-01T19:43:58.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:58 smithi087 ceph-mon[109386]: osdmap e69: 8 total, 8 up, 8 in 2023-10-01T19:43:59.614 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:59 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:43:59] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:43:59.614 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:59 smithi057 ceph-mon[108286]: osdmap e70: 8 total, 8 up, 8 in 2023-10-01T19:43:59.614 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:43:59 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:59.614 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:59 smithi057 ceph-mon[114331]: osdmap e70: 8 total, 8 up, 8 in 2023-10-01T19:43:59.614 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:43:59 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:59.746 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:59 smithi087 ceph-mon[109386]: osdmap e70: 8 total, 8 up, 8 in 2023-10-01T19:43:59.746 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:43:59 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:43:59.828 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch apply iscsi foo u p' 2023-10-01T19:43:59.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:59 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:43:59] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:44:00.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:43:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:43:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:44:00.222 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:43:59 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:43:59] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:44:00.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-mon[108286]: pgmap v62: 161 pgs: 161 active+clean; 584 KiB data, 53 MiB used, 715 GiB / 715 GiB avail; 116 KiB/s rd, 8.2 KiB/s wr, 280 op/s 2023-10-01T19:44:00.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-mon[108286]: osdmap e71: 8 total, 8 up, 8 in 2023-10-01T19:44:00.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:00.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:00.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:00.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:00.547 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-mon[114331]: pgmap v62: 161 pgs: 161 active+clean; 584 KiB data, 53 MiB used, 715 GiB / 715 GiB avail; 116 KiB/s rd, 8.2 KiB/s wr, 280 op/s 2023-10-01T19:44:00.547 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-mon[114331]: osdmap e71: 8 total, 8 up, 8 in 2023-10-01T19:44:00.547 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:00.547 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:00.547 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:00.547 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:00 smithi087 ceph-mon[109386]: pgmap v62: 161 pgs: 161 active+clean; 584 KiB data, 53 MiB used, 715 GiB / 715 GiB avail; 116 KiB/s rd, 8.2 KiB/s wr, 280 op/s 2023-10-01T19:44:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:00 smithi087 ceph-mon[109386]: osdmap e71: 8 total, 8 up, 8 in 2023-10-01T19:44:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:00 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:00 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:00 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:00 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:00.916 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:44:00] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:44:01.350 INFO:teuthology.orchestra.run.smithi057.stdout:Scheduled iscsi.foo update... 2023-10-01T19:44:01.724 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'sleep 180' 2023-10-01T19:44:01.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:01 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:01 smithi087 ceph-mon[109386]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 148 KiB/s rd, 1.0 KiB/s wr, 222 op/s 2023-10-01T19:44:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:01 smithi087 ceph-mon[109386]: from='client.24694 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:44:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:01 smithi087 ceph-mon[109386]: Saving service iscsi.foo spec with placement count:1 2023-10-01T19:44:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:01 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:01 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:01 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:01 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:01 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:01 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:44:01.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:01.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[108286]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 148 KiB/s rd, 1.0 KiB/s wr, 222 op/s 2023-10-01T19:44:01.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[108286]: from='client.24694 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:44:01.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[108286]: Saving service iscsi.foo spec with placement count:1 2023-10-01T19:44:01.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:01.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:01.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:01.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:01.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:01.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:44:01.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:01.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[114331]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 148 KiB/s rd, 1.0 KiB/s wr, 222 op/s 2023-10-01T19:44:01.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[114331]: from='client.24694 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:44:01.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[114331]: Saving service iscsi.foo spec with placement count:1 2023-10-01T19:44:01.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:01.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:01.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:01.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:01.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:01.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:01 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:44:04.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:04 smithi087 ceph-mon[109386]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 120 KiB/s rd, 853 B/s wr, 180 op/s 2023-10-01T19:44:04.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:04 smithi057 ceph-mon[108286]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 120 KiB/s rd, 853 B/s wr, 180 op/s 2023-10-01T19:44:04.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:04 smithi057 ceph-mon[114331]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 120 KiB/s rd, 853 B/s wr, 180 op/s 2023-10-01T19:44:04.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:04 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:44:04] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:44:05.628 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:05.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:05.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:05.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:05.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:05.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:05.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: Checking pool "foo" exists for service iscsi.foo 2023-10-01T19:44:05.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T19:44:05.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2023-10-01T19:44:05.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:05.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: Deploying daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T19:44:05.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: osdmap e72: 8 total, 8 up, 8 in 2023-10-01T19:44:05.630 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:05.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:05.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:05.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:05.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:05.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:05.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:05.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: Checking pool "foo" exists for service iscsi.foo 2023-10-01T19:44:05.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T19:44:05.631 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2023-10-01T19:44:05.631 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:05.631 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: Deploying daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T19:44:05.631 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: osdmap e72: 8 total, 8 up, 8 in 2023-10-01T19:44:05.631 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:05 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:05.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: Checking pool "foo" exists for service iscsi.foo 2023-10-01T19:44:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T19:44:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2023-10-01T19:44:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:05.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: Deploying daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T19:44:05.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: osdmap e72: 8 total, 8 up, 8 in 2023-10-01T19:44:05.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:05 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:06.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:06 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:44:06] "GET /metrics HTTP/1.1" 200 207121 "" "Prometheus/2.33.4" 2023-10-01T19:44:06.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:06 smithi057 ceph-mon[108286]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 162 op/s 2023-10-01T19:44:06.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:06 smithi057 ceph-mon[108286]: mgrmap e21: y(active, since 81s), standbys: x 2023-10-01T19:44:06.495 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:06 smithi057 ceph-mon[114331]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 162 op/s 2023-10-01T19:44:06.495 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:06 smithi057 ceph-mon[114331]: mgrmap e21: y(active, since 81s), standbys: x 2023-10-01T19:44:06.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:06 smithi087 ceph-mon[109386]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 162 op/s 2023-10-01T19:44:06.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:06 smithi087 ceph-mon[109386]: mgrmap e21: y(active, since 81s), standbys: x 2023-10-01T19:44:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:44:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:44:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:44:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: Adding iSCSI gateway http://:@172.21.15.57:5000 to Dashboard 2023-10-01T19:44:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:44:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:44:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:44:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:44:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:44:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s rd, 639 B/s wr, 135 op/s 2023-10-01T19:44:07.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3633783273' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T19:44:07.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3319231052' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2547329209"}]: dispatch 2023-10-01T19:44:07.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2547329209"}]: dispatch 2023-10-01T19:44:07.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:07.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:07.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:07.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:07.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:07.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:44:07.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:44:07.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:44:07.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: Adding iSCSI gateway http://:@172.21.15.57:5000 to Dashboard 2023-10-01T19:44:07.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:44:07.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:44:07.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:07.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:44:07.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:44:07.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:07.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:07.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:07.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:07.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:07.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:44:07.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s rd, 639 B/s wr, 135 op/s 2023-10-01T19:44:07.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3633783273' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T19:44:07.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3319231052' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2547329209"}]: dispatch 2023-10-01T19:44:07.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:07 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2547329209"}]: dispatch 2023-10-01T19:44:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:44:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:44:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:44:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: Adding iSCSI gateway http://:@172.21.15.57:5000 to Dashboard 2023-10-01T19:44:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:44:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:44:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:44:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:44:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:44:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s rd, 639 B/s wr, 135 op/s 2023-10-01T19:44:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3633783273' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T19:44:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3319231052' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2547329209"}]: dispatch 2023-10-01T19:44:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:07 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2547329209"}]: dispatch 2023-10-01T19:44:08.709 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:08 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2547329209"}]': finished 2023-10-01T19:44:08.709 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:08 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2547329209"}]': finished 2023-10-01T19:44:08.709 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:08 smithi057 ceph-mon[114331]: osdmap e73: 8 total, 8 up, 8 in 2023-10-01T19:44:08.710 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:08 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3776326074' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1285618755"}]: dispatch 2023-10-01T19:44:09.028 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:08 smithi057 ceph-mon[108286]: osdmap e73: 8 total, 8 up, 8 in 2023-10-01T19:44:09.028 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:08 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3776326074' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1285618755"}]: dispatch 2023-10-01T19:44:09.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:08 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2547329209"}]': finished 2023-10-01T19:44:09.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:08 smithi087 ceph-mon[109386]: osdmap e73: 8 total, 8 up, 8 in 2023-10-01T19:44:09.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:08 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3776326074' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1285618755"}]: dispatch 2023-10-01T19:44:09.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:44:09] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:44:10.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:44:09 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:44:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:44:10.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:09 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3776326074' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1285618755"}]': finished 2023-10-01T19:44:10.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:09 smithi087 ceph-mon[109386]: osdmap e74: 8 total, 8 up, 8 in 2023-10-01T19:44:10.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:09 smithi087 ceph-mon[109386]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:44:10.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:09 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2031951185' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3512058905"}]: dispatch 2023-10-01T19:44:10.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:09 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3512058905"}]: dispatch 2023-10-01T19:44:10.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:09 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:10.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3776326074' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1285618755"}]': finished 2023-10-01T19:44:10.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-mon[108286]: osdmap e74: 8 total, 8 up, 8 in 2023-10-01T19:44:10.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-mon[108286]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:44:10.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2031951185' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3512058905"}]: dispatch 2023-10-01T19:44:10.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3512058905"}]: dispatch 2023-10-01T19:44:10.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:10.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3776326074' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1285618755"}]': finished 2023-10-01T19:44:10.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-mon[114331]: osdmap e74: 8 total, 8 up, 8 in 2023-10-01T19:44:10.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-mon[114331]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:44:10.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2031951185' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3512058905"}]: dispatch 2023-10-01T19:44:10.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3512058905"}]: dispatch 2023-10-01T19:44:10.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:09 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:11.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:10 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3512058905"}]': finished 2023-10-01T19:44:11.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:10 smithi087 ceph-mon[109386]: osdmap e75: 8 total, 8 up, 8 in 2023-10-01T19:44:11.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:10 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:11.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:10 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3841711906' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1623382278"}]: dispatch 2023-10-01T19:44:11.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:10 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1623382278"}]: dispatch 2023-10-01T19:44:11.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:10 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3512058905"}]': finished 2023-10-01T19:44:11.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:10 smithi057 ceph-mon[108286]: osdmap e75: 8 total, 8 up, 8 in 2023-10-01T19:44:11.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:10 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:11.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:10 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3841711906' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1623382278"}]: dispatch 2023-10-01T19:44:11.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:10 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1623382278"}]: dispatch 2023-10-01T19:44:11.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:10 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:44:10] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:44:11.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:10 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3512058905"}]': finished 2023-10-01T19:44:11.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:10 smithi057 ceph-mon[114331]: osdmap e75: 8 total, 8 up, 8 in 2023-10-01T19:44:11.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:10 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:11.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:10 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3841711906' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1623382278"}]: dispatch 2023-10-01T19:44:11.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:10 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1623382278"}]: dispatch 2023-10-01T19:44:12.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:11 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1623382278"}]': finished 2023-10-01T19:44:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:11 smithi087 ceph-mon[109386]: osdmap e76: 8 total, 8 up, 8 in 2023-10-01T19:44:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:11 smithi087 ceph-mon[109386]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 3 op/s 2023-10-01T19:44:12.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:11 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3055086486' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/427084745"}]: dispatch 2023-10-01T19:44:12.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:11 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/427084745"}]: dispatch 2023-10-01T19:44:12.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1623382278"}]': finished 2023-10-01T19:44:12.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[108286]: osdmap e76: 8 total, 8 up, 8 in 2023-10-01T19:44:12.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:12.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:12.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:12.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:12.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:12.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:12.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[108286]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 3 op/s 2023-10-01T19:44:12.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3055086486' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/427084745"}]: dispatch 2023-10-01T19:44:12.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/427084745"}]: dispatch 2023-10-01T19:44:12.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1623382278"}]': finished 2023-10-01T19:44:12.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[114331]: osdmap e76: 8 total, 8 up, 8 in 2023-10-01T19:44:12.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:12.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:12.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:44:12.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:44:12.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:44:12.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:12.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[114331]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 3 op/s 2023-10-01T19:44:12.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3055086486' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/427084745"}]: dispatch 2023-10-01T19:44:12.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:11 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/427084745"}]: dispatch 2023-10-01T19:44:13.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:12 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/427084745"}]': finished 2023-10-01T19:44:13.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:12 smithi057 ceph-mon[114331]: osdmap e77: 8 total, 8 up, 8 in 2023-10-01T19:44:13.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:12 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2978625483' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/589184120"}]: dispatch 2023-10-01T19:44:13.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:12 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/427084745"}]': finished 2023-10-01T19:44:13.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:12 smithi057 ceph-mon[108286]: osdmap e77: 8 total, 8 up, 8 in 2023-10-01T19:44:13.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:12 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2978625483' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/589184120"}]: dispatch 2023-10-01T19:44:13.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:12 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/427084745"}]': finished 2023-10-01T19:44:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:12 smithi087 ceph-mon[109386]: osdmap e77: 8 total, 8 up, 8 in 2023-10-01T19:44:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:12 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2978625483' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/589184120"}]: dispatch 2023-10-01T19:44:13.917 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:44:13.522Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:44:13.917 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:44:13.522Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:44:14.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:13 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2978625483' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/589184120"}]': finished 2023-10-01T19:44:14.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:13 smithi057 ceph-mon[108286]: osdmap e78: 8 total, 8 up, 8 in 2023-10-01T19:44:14.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:13 smithi057 ceph-mon[108286]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 3 op/s 2023-10-01T19:44:14.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:13 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2555187828' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/19155294"}]: dispatch 2023-10-01T19:44:14.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:13 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2978625483' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/589184120"}]': finished 2023-10-01T19:44:14.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:13 smithi057 ceph-mon[114331]: osdmap e78: 8 total, 8 up, 8 in 2023-10-01T19:44:14.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:13 smithi057 ceph-mon[114331]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 3 op/s 2023-10-01T19:44:14.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:13 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2555187828' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/19155294"}]: dispatch 2023-10-01T19:44:14.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:13 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2978625483' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/589184120"}]': finished 2023-10-01T19:44:14.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:13 smithi087 ceph-mon[109386]: osdmap e78: 8 total, 8 up, 8 in 2023-10-01T19:44:14.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:13 smithi087 ceph-mon[109386]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 3 op/s 2023-10-01T19:44:14.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:13 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2555187828' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/19155294"}]: dispatch 2023-10-01T19:44:15.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:14 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2555187828' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/19155294"}]': finished 2023-10-01T19:44:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:14 smithi087 ceph-mon[109386]: osdmap e79: 8 total, 8 up, 8 in 2023-10-01T19:44:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:14 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/831899443' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/3618988359"}]: dispatch 2023-10-01T19:44:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:14 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/3618988359"}]: dispatch 2023-10-01T19:44:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:14 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2555187828' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/19155294"}]': finished 2023-10-01T19:44:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:14 smithi057 ceph-mon[108286]: osdmap e79: 8 total, 8 up, 8 in 2023-10-01T19:44:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:14 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/831899443' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/3618988359"}]: dispatch 2023-10-01T19:44:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:14 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/3618988359"}]: dispatch 2023-10-01T19:44:15.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:14 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2555187828' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/19155294"}]': finished 2023-10-01T19:44:15.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:14 smithi057 ceph-mon[114331]: osdmap e79: 8 total, 8 up, 8 in 2023-10-01T19:44:15.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:14 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/831899443' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/3618988359"}]: dispatch 2023-10-01T19:44:15.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:14 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/3618988359"}]: dispatch 2023-10-01T19:44:16.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:15 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/3618988359"}]': finished 2023-10-01T19:44:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:15 smithi087 ceph-mon[109386]: osdmap e80: 8 total, 8 up, 8 in 2023-10-01T19:44:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:15 smithi087 ceph-mon[109386]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:44:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:15 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2652326226' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1453123188"}]: dispatch 2023-10-01T19:44:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:15 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1453123188"}]: dispatch 2023-10-01T19:44:16.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:44:16] "GET /metrics HTTP/1.1" 200 208021 "" "Prometheus/2.33.4" 2023-10-01T19:44:16.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:15 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/3618988359"}]': finished 2023-10-01T19:44:16.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:15 smithi057 ceph-mon[108286]: osdmap e80: 8 total, 8 up, 8 in 2023-10-01T19:44:16.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:15 smithi057 ceph-mon[108286]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:44:16.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:15 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2652326226' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1453123188"}]: dispatch 2023-10-01T19:44:16.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:15 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1453123188"}]: dispatch 2023-10-01T19:44:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:15 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/3618988359"}]': finished 2023-10-01T19:44:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:15 smithi057 ceph-mon[114331]: osdmap e80: 8 total, 8 up, 8 in 2023-10-01T19:44:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:15 smithi057 ceph-mon[114331]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:44:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:15 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2652326226' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1453123188"}]: dispatch 2023-10-01T19:44:16.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:15 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1453123188"}]: dispatch 2023-10-01T19:44:17.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:16 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1453123188"}]': finished 2023-10-01T19:44:17.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:16 smithi057 ceph-mon[108286]: osdmap e81: 8 total, 8 up, 8 in 2023-10-01T19:44:17.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:16 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2096674845' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/34608214"}]: dispatch 2023-10-01T19:44:17.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:16 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/34608214"}]: dispatch 2023-10-01T19:44:17.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:16 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1453123188"}]': finished 2023-10-01T19:44:17.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:16 smithi057 ceph-mon[114331]: osdmap e81: 8 total, 8 up, 8 in 2023-10-01T19:44:17.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:16 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2096674845' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/34608214"}]: dispatch 2023-10-01T19:44:17.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:16 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/34608214"}]: dispatch 2023-10-01T19:44:17.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:16 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1453123188"}]': finished 2023-10-01T19:44:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:16 smithi087 ceph-mon[109386]: osdmap e81: 8 total, 8 up, 8 in 2023-10-01T19:44:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:16 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2096674845' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/34608214"}]: dispatch 2023-10-01T19:44:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:16 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/34608214"}]: dispatch 2023-10-01T19:44:18.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:17 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:17 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/34608214"}]': finished 2023-10-01T19:44:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:17 smithi087 ceph-mon[109386]: osdmap e82: 8 total, 8 up, 8 in 2023-10-01T19:44:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:17 smithi087 ceph-mon[109386]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:17 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3557239231' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2189167615"}]: dispatch 2023-10-01T19:44:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:17 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2189167615"}]: dispatch 2023-10-01T19:44:18.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:18 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:18.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:18 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/34608214"}]': finished 2023-10-01T19:44:18.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:18 smithi057 ceph-mon[108286]: osdmap e82: 8 total, 8 up, 8 in 2023-10-01T19:44:18.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:18 smithi057 ceph-mon[108286]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:18.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:18 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3557239231' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2189167615"}]: dispatch 2023-10-01T19:44:18.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:18 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2189167615"}]: dispatch 2023-10-01T19:44:18.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:18 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:18.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:18 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/34608214"}]': finished 2023-10-01T19:44:18.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:18 smithi057 ceph-mon[114331]: osdmap e82: 8 total, 8 up, 8 in 2023-10-01T19:44:18.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:18 smithi057 ceph-mon[114331]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:18.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:18 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3557239231' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2189167615"}]: dispatch 2023-10-01T19:44:18.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:18 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2189167615"}]: dispatch 2023-10-01T19:44:19.291 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:19 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2189167615"}]': finished 2023-10-01T19:44:19.291 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:19 smithi087 ceph-mon[109386]: osdmap e83: 8 total, 8 up, 8 in 2023-10-01T19:44:19.291 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:19 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1406618834' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/3618988359"}]: dispatch 2023-10-01T19:44:19.291 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:19 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/3618988359"}]: dispatch 2023-10-01T19:44:19.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:19 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2189167615"}]': finished 2023-10-01T19:44:19.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:19 smithi057 ceph-mon[108286]: osdmap e83: 8 total, 8 up, 8 in 2023-10-01T19:44:19.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:19 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1406618834' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/3618988359"}]: dispatch 2023-10-01T19:44:19.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:19 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/3618988359"}]: dispatch 2023-10-01T19:44:19.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:19 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2189167615"}]': finished 2023-10-01T19:44:19.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:19 smithi057 ceph-mon[114331]: osdmap e83: 8 total, 8 up, 8 in 2023-10-01T19:44:19.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:19 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1406618834' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/3618988359"}]: dispatch 2023-10-01T19:44:19.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:19 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/3618988359"}]: dispatch 2023-10-01T19:44:20.192 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:44:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:44:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:44:20.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:20 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/3618988359"}]': finished 2023-10-01T19:44:20.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:20 smithi087 ceph-mon[109386]: osdmap e84: 8 total, 8 up, 8 in 2023-10-01T19:44:20.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:20 smithi087 ceph-mon[109386]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:20.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:20 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1175034893' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/1407753143"}]: dispatch 2023-10-01T19:44:20.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:20 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/3618988359"}]': finished 2023-10-01T19:44:20.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:20 smithi057 ceph-mon[108286]: osdmap e84: 8 total, 8 up, 8 in 2023-10-01T19:44:20.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:20 smithi057 ceph-mon[108286]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:20.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:20 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1175034893' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/1407753143"}]: dispatch 2023-10-01T19:44:20.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:20 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/3618988359"}]': finished 2023-10-01T19:44:20.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:20 smithi057 ceph-mon[114331]: osdmap e84: 8 total, 8 up, 8 in 2023-10-01T19:44:20.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:20 smithi057 ceph-mon[114331]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:20.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:20 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1175034893' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/1407753143"}]: dispatch 2023-10-01T19:44:20.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:20 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:44:20] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:44:21.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:21 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1175034893' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/1407753143"}]': finished 2023-10-01T19:44:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:21 smithi087 ceph-mon[109386]: osdmap e85: 8 total, 8 up, 8 in 2023-10-01T19:44:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:21 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2859133814' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2547329209"}]: dispatch 2023-10-01T19:44:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:21 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2547329209"}]: dispatch 2023-10-01T19:44:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:21 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:21 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:21.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:21 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1175034893' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/1407753143"}]': finished 2023-10-01T19:44:21.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:21 smithi057 ceph-mon[108286]: osdmap e85: 8 total, 8 up, 8 in 2023-10-01T19:44:21.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:21 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2859133814' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2547329209"}]: dispatch 2023-10-01T19:44:21.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:21 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2547329209"}]: dispatch 2023-10-01T19:44:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:21 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:21 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:21 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1175034893' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/1407753143"}]': finished 2023-10-01T19:44:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:21 smithi057 ceph-mon[114331]: osdmap e85: 8 total, 8 up, 8 in 2023-10-01T19:44:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:21 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2859133814' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2547329209"}]: dispatch 2023-10-01T19:44:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:21 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2547329209"}]: dispatch 2023-10-01T19:44:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:21 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:21 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:22.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:22 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2547329209"}]': finished 2023-10-01T19:44:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:22 smithi087 ceph-mon[109386]: osdmap e86: 8 total, 8 up, 8 in 2023-10-01T19:44:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:22 smithi087 ceph-mon[109386]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 62 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:22 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2316256881' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/69774414"}]: dispatch 2023-10-01T19:44:22.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:22 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2547329209"}]': finished 2023-10-01T19:44:22.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:22 smithi057 ceph-mon[114331]: osdmap e86: 8 total, 8 up, 8 in 2023-10-01T19:44:22.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:22 smithi057 ceph-mon[114331]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 62 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:22.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:22 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2316256881' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/69774414"}]: dispatch 2023-10-01T19:44:22.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:22 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2547329209"}]': finished 2023-10-01T19:44:22.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:22 smithi057 ceph-mon[108286]: osdmap e86: 8 total, 8 up, 8 in 2023-10-01T19:44:22.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:22 smithi057 ceph-mon[108286]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 62 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:22.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:22 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2316256881' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/69774414"}]: dispatch 2023-10-01T19:44:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:23 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2316256881' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/69774414"}]': finished 2023-10-01T19:44:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:23 smithi087 ceph-mon[109386]: osdmap e87: 8 total, 8 up, 8 in 2023-10-01T19:44:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:23 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1115632998' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/1407753143"}]: dispatch 2023-10-01T19:44:23.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:23 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2316256881' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/69774414"}]': finished 2023-10-01T19:44:23.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:23 smithi057 ceph-mon[108286]: osdmap e87: 8 total, 8 up, 8 in 2023-10-01T19:44:23.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:23 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1115632998' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/1407753143"}]: dispatch 2023-10-01T19:44:23.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:23 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2316256881' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/69774414"}]': finished 2023-10-01T19:44:23.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:23 smithi057 ceph-mon[114331]: osdmap e87: 8 total, 8 up, 8 in 2023-10-01T19:44:23.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:23 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1115632998' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/1407753143"}]: dispatch 2023-10-01T19:44:23.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:44:23.507Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:44:23.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:44:23.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:44:23.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:44:23.509Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:44:24.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:24 smithi087 ceph-mon[109386]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 62 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:24 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1115632998' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/1407753143"}]': finished 2023-10-01T19:44:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:24 smithi087 ceph-mon[109386]: osdmap e88: 8 total, 8 up, 8 in 2023-10-01T19:44:24.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:24 smithi057 ceph-mon[108286]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 62 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:24.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:24 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1115632998' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/1407753143"}]': finished 2023-10-01T19:44:24.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:24 smithi057 ceph-mon[108286]: osdmap e88: 8 total, 8 up, 8 in 2023-10-01T19:44:24.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:24 smithi057 ceph-mon[114331]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 62 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:24.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:24 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1115632998' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/1407753143"}]': finished 2023-10-01T19:44:24.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:24 smithi057 ceph-mon[114331]: osdmap e88: 8 total, 8 up, 8 in 2023-10-01T19:44:26.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:25 smithi057 ceph-mon[108286]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 62 MiB used, 715 GiB / 715 GiB avail; 1015 B/s rd, 0 op/s 2023-10-01T19:44:26.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:25 smithi057 ceph-mon[114331]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 62 MiB used, 715 GiB / 715 GiB avail; 1015 B/s rd, 0 op/s 2023-10-01T19:44:26.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:25 smithi087 ceph-mon[109386]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 62 MiB used, 715 GiB / 715 GiB avail; 1015 B/s rd, 0 op/s 2023-10-01T19:44:26.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:44:26] "GET /metrics HTTP/1.1" 200 208021 "" "Prometheus/2.33.4" 2023-10-01T19:44:28.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:28 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:28.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:28 smithi087 ceph-mon[109386]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2023-10-01T19:44:28.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:28 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:28.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:28 smithi057 ceph-mon[108286]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2023-10-01T19:44:28.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:28 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:28.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:28 smithi057 ceph-mon[114331]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2023-10-01T19:44:29.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:29 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:44:29] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:44:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:30 smithi087 ceph-mon[109386]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2023-10-01T19:44:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:30.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:44:29 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:44:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:44:30.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:30 smithi057 ceph-mon[114331]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2023-10-01T19:44:30.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:30.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:30 smithi057 ceph-mon[108286]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2023-10-01T19:44:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:32.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:32 smithi087 ceph-mon[109386]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T19:44:32.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:32 smithi057 ceph-mon[108286]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T19:44:32.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:32 smithi057 ceph-mon[114331]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T19:44:33.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:44:33.508Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:44:33.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:44:33.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:44:33.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:44:33.511Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:44:34.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:34 smithi087 ceph-mon[109386]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-10-01T19:44:34.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:34 smithi057 ceph-mon[108286]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-10-01T19:44:34.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:34 smithi057 ceph-mon[114331]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-10-01T19:44:36.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:35 smithi057 ceph-mon[108286]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 855 B/s rd, 0 op/s 2023-10-01T19:44:36.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:35 smithi057 ceph-mon[114331]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 855 B/s rd, 0 op/s 2023-10-01T19:44:36.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:35 smithi087 ceph-mon[109386]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 855 B/s rd, 0 op/s 2023-10-01T19:44:36.422 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:36 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:44:36] "GET /metrics HTTP/1.1" 200 208037 "" "Prometheus/2.33.4" 2023-10-01T19:44:38.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:38 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:38.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:38 smithi087 ceph-mon[109386]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:38.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:38 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:38.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:38 smithi057 ceph-mon[108286]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:38.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:38 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:38.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:38 smithi057 ceph-mon[114331]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:40.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:40 smithi087 ceph-mon[109386]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:44:40.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:44:39 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:44:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:44:40.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:40 smithi057 ceph-mon[108286]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:44:40.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:40 smithi057 ceph-mon[114331]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:44:41.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:40 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:44:40] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:44:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:41 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:41 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:41 smithi087 ceph-mon[109386]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:42.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:41 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:42.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:41 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:42.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:41 smithi057 ceph-mon[108286]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:42.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:41 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:42.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:41 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:42.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:41 smithi057 ceph-mon[114331]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:43.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:44:43.509Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:44:43.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:44:43.511Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:44:43.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:44:43.512Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:44:44.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:44 smithi087 ceph-mon[109386]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:44:44.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:44 smithi057 ceph-mon[108286]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:44:44.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:44 smithi057 ceph-mon[114331]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:44:45.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2023-10-01T19:44:45.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2023-10-01T19:44:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2023-10-01T19:44:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2023-10-01T19:44:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:44:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:44:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2023-10-01T19:44:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2023-10-01T19:44:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2023-10-01T19:44:45.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2023-10-01T19:44:45.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:44:45.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:44:45.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2023-10-01T19:44:45.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2023-10-01T19:44:45.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2023-10-01T19:44:45.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2023-10-01T19:44:45.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:44:45.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:44:46.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:44:46] "GET /metrics HTTP/1.1" 200 208001 "" "Prometheus/2.33.4" 2023-10-01T19:44:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-mon[108286]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:44:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]': finished 2023-10-01T19:44:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]': finished 2023-10-01T19:44:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2023-10-01T19:44:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2023-10-01T19:44:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-mon[108286]: osdmap e89: 8 total, 8 up, 8 in 2023-10-01T19:44:46.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-mon[114331]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:44:46.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]': finished 2023-10-01T19:44:46.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]': finished 2023-10-01T19:44:46.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2023-10-01T19:44:46.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2023-10-01T19:44:46.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:46 smithi057 ceph-mon[114331]: osdmap e89: 8 total, 8 up, 8 in 2023-10-01T19:44:46.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:46 smithi087 ceph-mon[109386]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:44:46.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:46 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]': finished 2023-10-01T19:44:46.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:46 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]': finished 2023-10-01T19:44:46.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:46 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2023-10-01T19:44:46.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:46 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2023-10-01T19:44:46.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:46 smithi087 ceph-mon[109386]: osdmap e89: 8 total, 8 up, 8 in 2023-10-01T19:44:47.460 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:47 smithi057 ceph-mon[108286]: osdmap e90: 8 total, 8 up, 8 in 2023-10-01T19:44:47.460 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:47 smithi057 ceph-mon[114331]: osdmap e90: 8 total, 8 up, 8 in 2023-10-01T19:44:47.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:47 smithi087 ceph-mon[109386]: osdmap e90: 8 total, 8 up, 8 in 2023-10-01T19:44:48.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:48 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:48.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:48 smithi057 ceph-mon[108286]: pgmap v106: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:48.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:48 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:48.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:48 smithi057 ceph-mon[114331]: pgmap v106: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:48.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:48 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:48 smithi087 ceph-mon[109386]: pgmap v106: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:44:49.346 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:44:49] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:44:50.216 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:44:49 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:44:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:44:50.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:50 smithi087 ceph-mon[109386]: pgmap v107: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2023-10-01T19:44:50.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:50 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:50.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:50 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:50.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:50 smithi057 ceph-mon[108286]: pgmap v107: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2023-10-01T19:44:50.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:50 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:50.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:50 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:50.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:50 smithi057 ceph-mon[114331]: pgmap v107: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 63 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2023-10-01T19:44:50.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:50 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:50.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:50 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:44:52.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:52 smithi087 ceph-mon[109386]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 50 B/s, 0 objects/s recovering 2023-10-01T19:44:52.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:52 smithi057 ceph-mon[108286]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 50 B/s, 0 objects/s recovering 2023-10-01T19:44:52.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:52 smithi057 ceph-mon[114331]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 50 B/s, 0 objects/s recovering 2023-10-01T19:44:53.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:44:53.510Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:44:53.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:44:53.512Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:44:53.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:44:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:44:53.512Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:44:54.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:54 smithi087 ceph-mon[109386]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 50 B/s, 0 objects/s recovering 2023-10-01T19:44:54.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:54 smithi057 ceph-mon[108286]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 50 B/s, 0 objects/s recovering 2023-10-01T19:44:54.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:54 smithi057 ceph-mon[114331]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 50 B/s, 0 objects/s recovering 2023-10-01T19:44:56.057 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:55 smithi057 ceph-mon[108286]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 40 B/s, 0 objects/s recovering 2023-10-01T19:44:56.057 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:55 smithi057 ceph-mon[114331]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 40 B/s, 0 objects/s recovering 2023-10-01T19:44:56.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:55 smithi087 ceph-mon[109386]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 40 B/s, 0 objects/s recovering 2023-10-01T19:44:56.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:44:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:44:56] "GET /metrics HTTP/1.1" 200 208001 "" "Prometheus/2.33.4" 2023-10-01T19:44:58.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:58 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:44:58 smithi087 ceph-mon[109386]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s; 37 B/s, 0 objects/s recovering 2023-10-01T19:44:58.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:58 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:58.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:44:58 smithi057 ceph-mon[108286]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s; 37 B/s, 0 objects/s recovering 2023-10-01T19:44:58.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:58 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:44:58.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:44:58 smithi057 ceph-mon[114331]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s; 37 B/s, 0 objects/s recovering 2023-10-01T19:45:00.131 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:44:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:44:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:45:00.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:00 smithi087 ceph-mon[109386]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 33 B/s, 0 objects/s recovering 2023-10-01T19:45:00.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:00 smithi057 ceph-mon[108286]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 33 B/s, 0 objects/s recovering 2023-10-01T19:45:00.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:00 smithi057 ceph-mon[114331]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 33 B/s, 0 objects/s recovering 2023-10-01T19:45:01.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:45:00 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:45:00] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:45:02.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:01 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:02.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:01 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:02.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:01 smithi087 ceph-mon[109386]: pgmap v113: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 33 B/s, 0 objects/s recovering 2023-10-01T19:45:02.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:01 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:02.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:01 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:02.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:01 smithi057 ceph-mon[108286]: pgmap v113: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 33 B/s, 0 objects/s recovering 2023-10-01T19:45:02.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:01 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:02.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:01 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:02.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:01 smithi057 ceph-mon[114331]: pgmap v113: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 33 B/s, 0 objects/s recovering 2023-10-01T19:45:03.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:45:03.511Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:45:03.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:45:03.514Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:45:03.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:45:03.514Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:45:04.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:04 smithi057 ceph-mon[108286]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:04.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:04 smithi057 ceph-mon[114331]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:04.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:04 smithi087 ceph-mon[109386]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:06.057 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:05 smithi057 ceph-mon[108286]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:06.057 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:05 smithi057 ceph-mon[114331]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:06.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:05 smithi087 ceph-mon[109386]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:06.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:45:06 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:45:06] "GET /metrics HTTP/1.1" 200 208044 "" "Prometheus/2.33.4" 2023-10-01T19:45:08.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:08 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:08.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:08 smithi057 ceph-mon[108286]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:08.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:08 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:08.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:08 smithi057 ceph-mon[114331]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:08.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:08 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:08 smithi087 ceph-mon[109386]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:09.527 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:45:09 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:45:09] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:45:10.345 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:45:09 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:45:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:45:10.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:10 smithi087 ceph-mon[109386]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:10.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:10 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:10.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:10 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:10.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:10 smithi057 ceph-mon[114331]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:10.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:10 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:10.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:10 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:10.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:10 smithi057 ceph-mon[108286]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:10.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:10 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:10.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:10 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:45:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:45:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:45:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:11.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:45:11.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:45:11.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:45:11.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:11.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:45:11.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:45:11.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:45:11.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:12.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:12 smithi087 ceph-mon[109386]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:12 smithi057 ceph-mon[108286]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:12.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:12 smithi057 ceph-mon[114331]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:13.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:45:13.512Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:45:13.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:45:13.514Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:45:13.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:45:13.515Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:45:14.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:14 smithi087 ceph-mon[109386]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:14.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:14 smithi057 ceph-mon[108286]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:14.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:14 smithi057 ceph-mon[114331]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:15.995 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:15 smithi087 ceph-mon[109386]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:16.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:15 smithi057 ceph-mon[108286]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:16.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:15 smithi057 ceph-mon[114331]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:16.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:45:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:45:16] "GET /metrics HTTP/1.1" 200 208023 "" "Prometheus/2.33.4" 2023-10-01T19:45:18.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:18 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:18.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:18 smithi057 ceph-mon[108286]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:18.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:18 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:18.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:18 smithi057 ceph-mon[114331]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:18.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:18 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:18.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:18 smithi087 ceph-mon[109386]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:20.159 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:45:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:45:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:45:20.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:20 smithi057 ceph-mon[108286]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:20.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:20 smithi057 ceph-mon[114331]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:20.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:20 smithi087 ceph-mon[109386]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:21.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:45:20 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:45:20] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:45:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:21 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:21 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:21 smithi087 ceph-mon[109386]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:22.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:21 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:22.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:21 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:22.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:21 smithi057 ceph-mon[108286]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:22.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:21 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:22.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:21 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:22.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:21 smithi057 ceph-mon[114331]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:23.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:45:23.513Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:45:23.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:45:23.515Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:45:23.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:45:23.516Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:45:24.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:24 smithi057 ceph-mon[108286]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:24.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:24 smithi057 ceph-mon[114331]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:24.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:24 smithi087 ceph-mon[109386]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:26.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:25 smithi057 ceph-mon[108286]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:26.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:25 smithi057 ceph-mon[114331]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:26.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:25 smithi087 ceph-mon[109386]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:26.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:45:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:45:26] "GET /metrics HTTP/1.1" 200 208023 "" "Prometheus/2.33.4" 2023-10-01T19:45:28.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:28 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:28.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:28 smithi057 ceph-mon[108286]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:28.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:28 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:28.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:28 smithi057 ceph-mon[114331]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:28.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:28 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:28.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:28 smithi087 ceph-mon[109386]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:29.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:45:29 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:45:29] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:45:30.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:45:29 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:45:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:45:30.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:30 smithi087 ceph-mon[109386]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:30.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:30.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:30.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:30 smithi057 ceph-mon[108286]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:30.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:30.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:30.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:30 smithi057 ceph-mon[114331]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:30.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:30.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:32.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:32 smithi087 ceph-mon[109386]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:32.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:32 smithi057 ceph-mon[108286]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:32.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:32 smithi057 ceph-mon[114331]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:33.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:45:33.514Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:45:33.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:45:33.516Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:45:33.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:45:33.516Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:45:34.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:34 smithi087 ceph-mon[109386]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:34.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:34 smithi057 ceph-mon[108286]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:34.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:34 smithi057 ceph-mon[114331]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:36.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:35 smithi057 ceph-mon[108286]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:36.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:35 smithi057 ceph-mon[114331]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:36.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:35 smithi087 ceph-mon[109386]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:36.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:45:36 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:45:36] "GET /metrics HTTP/1.1" 200 207999 "" "Prometheus/2.33.4" 2023-10-01T19:45:38.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:38 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:38.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:38 smithi057 ceph-mon[108286]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:38.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:38 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:38.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:38 smithi057 ceph-mon[114331]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:38.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:38 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:38.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:38 smithi087 ceph-mon[109386]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:40.261 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:40 smithi087 ceph-mon[109386]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:40.262 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:45:39 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:45:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:45:40.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:40 smithi057 ceph-mon[108286]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:40.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:40 smithi057 ceph-mon[114331]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:41.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:45:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:45:41] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:45:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:42 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:42 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:42 smithi087 ceph-mon[109386]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:42.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:42 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:42.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:42 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:42.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:42 smithi057 ceph-mon[108286]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:42.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:42 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:42.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:42 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:42.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:42 smithi057 ceph-mon[114331]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:43.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:45:43.514Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:45:43.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:45:43.516Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:45:43.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:45:43.517Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:45:44.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:44 smithi057 ceph-mon[108286]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:44.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:44 smithi057 ceph-mon[114331]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:44.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:44 smithi087 ceph-mon[109386]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:46.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:45:46.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:45:46.059 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:45 smithi057 ceph-mon[108286]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:46.059 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:45:46.059 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:45:46.059 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:45 smithi057 ceph-mon[114331]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:46.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:45:46.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:45:46.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:45 smithi087 ceph-mon[109386]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:46.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:45:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:45:46] "GET /metrics HTTP/1.1" 200 207989 "" "Prometheus/2.33.4" 2023-10-01T19:45:48.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:48 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:48.412 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:48 smithi057 ceph-mon[108286]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:48.412 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:48 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:48.412 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:48 smithi057 ceph-mon[114331]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:48.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:48 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:48 smithi087 ceph-mon[109386]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:49.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:45:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:45:49] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:45:50.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:45:49 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:45:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:45:50.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:50 smithi087 ceph-mon[109386]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:50 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:50 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:50.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:50 smithi057 ceph-mon[108286]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:50.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:50 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:50.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:50 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:50.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:50 smithi057 ceph-mon[114331]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:50.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:50 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:50.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:50 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:45:52.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:52 smithi057 ceph-mon[108286]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:52.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:52 smithi057 ceph-mon[114331]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:53.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:52 smithi087 ceph-mon[109386]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:53.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:53 smithi057 ceph-mon[108286]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:53.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:53 smithi057 ceph-mon[114331]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:53.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:45:53.515Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:45:53.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:45:53.517Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:45:53.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:45:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:45:53.517Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:45:54.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:53 smithi087 ceph-mon[109386]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:56.057 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:55 smithi057 ceph-mon[108286]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:56.057 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:55 smithi057 ceph-mon[114331]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:56.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:55 smithi087 ceph-mon[109386]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:45:56.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:45:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:45:56] "GET /metrics HTTP/1.1" 200 207989 "" "Prometheus/2.33.4" 2023-10-01T19:45:58.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:58 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:58.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:45:58 smithi057 ceph-mon[108286]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:58.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:58 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:58.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:45:58 smithi057 ceph-mon[114331]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:45:58.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:58 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:45:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:45:58 smithi087 ceph-mon[109386]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:00.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:00 smithi087 ceph-mon[109386]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:00.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:45:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:45:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:46:00.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:00 smithi057 ceph-mon[114331]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:00.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:00 smithi057 ceph-mon[108286]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:01.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:46:01 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:46:01] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:46:02.504 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:02 smithi087 ceph-mon[109386]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:02.504 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:02 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:02.504 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:02 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:02.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:02 smithi057 ceph-mon[108286]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:02.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:02 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:02.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:02 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:02.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:02 smithi057 ceph-mon[114331]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:02.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:02 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:02.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:02 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:03.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:46:03.515Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:46:03.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:46:03.517Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:46:03.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:46:03.518Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:46:04.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:04 smithi087 ceph-mon[109386]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:04.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:04 smithi057 ceph-mon[108286]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:04.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:04 smithi057 ceph-mon[114331]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:06.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:05 smithi057 ceph-mon[108286]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:06.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:05 smithi057 ceph-mon[114331]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:06.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:05 smithi087 ceph-mon[109386]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:06.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:46:06 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:46:06] "GET /metrics HTTP/1.1" 200 208012 "" "Prometheus/2.33.4" 2023-10-01T19:46:08.366 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:08 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:08.366 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:08 smithi057 ceph-mon[108286]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:08.366 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:08 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:08.366 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:08 smithi057 ceph-mon[114331]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:08 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:08 smithi087 ceph-mon[109386]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:10.168 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:46:09 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:46:09] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:46:10.170 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:46:09 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:46:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:46:10.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:10 smithi057 ceph-mon[108286]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:10.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:10 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:10.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:10 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:10.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:10 smithi057 ceph-mon[114331]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:10.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:10 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:10.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:10 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:10.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:10 smithi087 ceph-mon[109386]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:10.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:10 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:10.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:10 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:46:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:46:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:46:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:11.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:46:11.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:46:11.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:46:11.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:11.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:46:11.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:46:11.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:46:11.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:12.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:12 smithi087 ceph-mon[109386]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:12 smithi057 ceph-mon[108286]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:12.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:12 smithi057 ceph-mon[114331]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:13.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:46:13.516Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:46:13.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:46:13.518Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:46:13.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:46:13.518Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:46:14.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:14 smithi087 ceph-mon[109386]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:14.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:14 smithi057 ceph-mon[108286]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:14.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:14 smithi057 ceph-mon[114331]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:16.057 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:15 smithi057 ceph-mon[108286]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:16.057 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:15 smithi057 ceph-mon[114331]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:16.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:15 smithi087 ceph-mon[109386]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:16.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:46:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:46:16] "GET /metrics HTTP/1.1" 200 207999 "" "Prometheus/2.33.4" 2023-10-01T19:46:18.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:18 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:18.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:18 smithi057 ceph-mon[108286]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:18.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:18 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:18.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:18 smithi057 ceph-mon[114331]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:18.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:18 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:18 smithi087 ceph-mon[109386]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:20.173 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:46:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:46:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:46:20.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:20 smithi087 ceph-mon[109386]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:20.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:20 smithi057 ceph-mon[108286]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:20.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:20 smithi057 ceph-mon[114331]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:21.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:46:21 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:46:21] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:46:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:22 smithi087 ceph-mon[109386]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:22 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:22 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:22.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:22 smithi057 ceph-mon[108286]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:22.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:22 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:22.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:22 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:22.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:22 smithi057 ceph-mon[114331]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:22.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:22 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:22.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:22 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:23.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:46:23.516Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:46:23.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:46:23.518Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:46:23.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:46:23.519Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:46:24.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:24 smithi087 ceph-mon[109386]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:24.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:24 smithi057 ceph-mon[108286]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:24.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:24 smithi057 ceph-mon[114331]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:26.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:25 smithi057 ceph-mon[108286]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:26.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:25 smithi057 ceph-mon[114331]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:26.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:25 smithi087 ceph-mon[109386]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:26.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:46:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:46:26] "GET /metrics HTTP/1.1" 200 207999 "" "Prometheus/2.33.4" 2023-10-01T19:46:28.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:28 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:28.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:28 smithi057 ceph-mon[108286]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:28.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:28 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:28.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:28 smithi057 ceph-mon[114331]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:28.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:28 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:28.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:28 smithi087 ceph-mon[109386]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:30.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:46:29 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:46:29] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:46:30.177 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:46:29 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:46:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:46:30.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:30 smithi057 ceph-mon[108286]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:30.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:30.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:30.448 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:30 smithi057 ceph-mon[114331]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:30.448 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:30.448 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:30 smithi087 ceph-mon[109386]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:32.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:32 smithi087 ceph-mon[109386]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:32.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:32 smithi057 ceph-mon[108286]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:32.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:32 smithi057 ceph-mon[114331]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:33.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:46:33.517Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:46:33.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:46:33.519Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:46:33.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:46:33.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:46:34.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:34 smithi087 ceph-mon[109386]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:34.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:34 smithi057 ceph-mon[108286]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:34.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:34 smithi057 ceph-mon[114331]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:36.057 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:35 smithi057 ceph-mon[108286]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:36.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:35 smithi057 ceph-mon[114331]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:36.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:35 smithi087 ceph-mon[109386]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:36.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:46:36 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:46:36] "GET /metrics HTTP/1.1" 200 207993 "" "Prometheus/2.33.4" 2023-10-01T19:46:38.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:38 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:38 smithi057 ceph-mon[108286]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:38.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:38 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:38.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:38 smithi057 ceph-mon[114331]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:38.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:38 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:38.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:38 smithi087 ceph-mon[109386]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:40.168 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:46:39 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:46:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:46:40.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:40 smithi057 ceph-mon[108286]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:40.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:40 smithi057 ceph-mon[114331]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:40.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:40 smithi087 ceph-mon[109386]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:41.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:46:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:46:41] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:46:42.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:42 smithi057 ceph-mon[108286]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:42.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:42 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:42.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:42 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:42 smithi057 ceph-mon[114331]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:42 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:42 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:42.725 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:42 smithi087 ceph-mon[109386]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:42.725 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:42 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:42.725 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:42 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:43.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:46:43.518Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:46:43.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:46:43.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:46:43.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:46:43.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:46:44.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:44 smithi087 ceph-mon[109386]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:44.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:44 smithi057 ceph-mon[108286]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:44.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:44 smithi057 ceph-mon[114331]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:46.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:46:46.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:46:46.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:45 smithi057 ceph-mon[108286]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:46.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:46:46.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:46:46.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:45 smithi057 ceph-mon[114331]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:46.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:46:46.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:46:46.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:45 smithi087 ceph-mon[109386]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:46.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:46:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:46:46] "GET /metrics HTTP/1.1" 200 207992 "" "Prometheus/2.33.4" 2023-10-01T19:46:48.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:48 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:48.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:48 smithi057 ceph-mon[108286]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:48.586 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:48 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:48.586 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:48 smithi057 ceph-mon[114331]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:48.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:48 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:48 smithi087 ceph-mon[109386]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:50.190 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:46:49 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:46:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:46:50.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:46:50 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:46:50] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:46:50.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:50 smithi057 ceph-mon[108286]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:50.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:50 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:50.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:50 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:50 smithi057 ceph-mon[114331]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:50 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:50 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:50.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:50 smithi087 ceph-mon[109386]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:50.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:50 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:50.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:50 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:46:52.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:52 smithi087 ceph-mon[109386]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:52.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:52 smithi057 ceph-mon[108286]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:52.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:52 smithi057 ceph-mon[114331]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:53.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:46:53.518Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:46:53.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:46:53.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:46:53.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:46:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:46:53.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:46:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:54 smithi087 ceph-mon[109386]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:54.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:54 smithi057 ceph-mon[108286]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:54.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:54 smithi057 ceph-mon[114331]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:56.059 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:55 smithi057 ceph-mon[108286]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:56.059 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:55 smithi057 ceph-mon[114331]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:56.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:55 smithi087 ceph-mon[109386]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:46:56.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:46:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:46:56] "GET /metrics HTTP/1.1" 200 207992 "" "Prometheus/2.33.4" 2023-10-01T19:46:58.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:58 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:58.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:46:58 smithi057 ceph-mon[114331]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:58.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:58 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:58.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:46:58 smithi057 ceph-mon[108286]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:46:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:58 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:46:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:46:58 smithi087 ceph-mon[109386]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:00.169 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:46:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:46:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:47:00.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:00 smithi057 ceph-mon[108286]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:00 smithi057 ceph-mon[114331]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:00 smithi087 ceph-mon[109386]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:01.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:47:01 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:47:01] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:47:02.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:02 smithi087 ceph-mon[109386]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:02.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:02 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:02.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:02 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:02.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:02 smithi057 ceph-mon[108286]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:02.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:02 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:02.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:02 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:02 smithi057 ceph-mon[114331]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:02 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:02 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:03.312 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2023-10-01T19:47:03.660 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:47:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:47:03.518Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:47:04.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:04 smithi057 ceph-mon[108286]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:04.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:04 smithi057 ceph-mon[114331]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:04.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:04 smithi087 ceph-mon[109386]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:05.178 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2023-10-01T19:47:05.917 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:05 smithi057 ceph-mon[108286]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:05.918 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:05 smithi057 ceph-mon[114331]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:06.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:05 smithi087 ceph-mon[109386]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:06.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:47:06 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:47:06] "GET /metrics HTTP/1.1" 200 207952 "" "Prometheus/2.33.4" 2023-10-01T19:47:06.944 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph config set global log_to_journald false --force' 2023-10-01T19:47:08.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:08 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:08.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:08 smithi057 ceph-mon[108286]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:08.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:08 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:08.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:08 smithi057 ceph-mon[114331]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:08.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:08 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:08 smithi087 ceph-mon[109386]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:08.815 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch ps' 2023-10-01T19:47:10.179 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:47:09 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:47:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:47:10.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:10 smithi057 ceph-mon[108286]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:10.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:10 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:10.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:10 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:10.423 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:47:10 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:47:10] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:47:10.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:10 smithi057 ceph-mon[114331]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:10.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:10 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:10.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:10 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:10.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:10 smithi087 ceph-mon[109386]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:10.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:10 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:10.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:10 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:10.705 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T19:47:10.705 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 0s ago 7m - - 2023-10-01T19:47:10.705 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 9s ago 7m - - 2023-10-01T19:47:10.705 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (3m) 0s ago 3m 29.4M - ba2b418f427c 1b7e399a166e 2023-10-01T19:47:10.705 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (3m) 9s ago 3m 60.9M - 8.3.5 dad864ee21e9 19c2251f4393 2023-10-01T19:47:10.705 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (3m) 0s ago 3m 56.5M - 3.5 e1d6a67b021e df9831dacebb 2023-10-01T19:47:10.705 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443 running (6m) 9s ago 6m 420M - 17.2.0 e1d6a67b021e d46b745be5ba 2023-10-01T19:47:10.705 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:9283 running (8m) 0s ago 8m 484M - 17.2.0 e1d6a67b021e d546d6fb718a 2023-10-01T19:47:10.705 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (8m) 0s ago 8m 97.2M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T19:47:10.706 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (7m) 9s ago 7m 61.7M 2048M 17.2.0 e1d6a67b021e 4734e639b775 2023-10-01T19:47:10.706 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (7m) 0s ago 7m 59.3M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T19:47:10.706 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (4m) 0s ago 4m 23.7M - 1dbe0e931976 efd276ccf21b 2023-10-01T19:47:10.706 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (4m) 9s ago 4m 20.2M - 1dbe0e931976 6d094569ea47 2023-10-01T19:47:10.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (6m) 0s ago 6m 87.7M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T19:47:10.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (6m) 0s ago 6m 93.1M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T19:47:10.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (6m) 0s ago 6m 73.1M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T19:47:10.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (5m) 0s ago 5m 89.9M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T19:47:10.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (5m) 9s ago 5m 87.4M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T19:47:10.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (5m) 9s ago 5m 86.5M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T19:47:10.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (5m) 9s ago 5m 73.3M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T19:47:10.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (4m) 9s ago 4m 86.9M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T19:47:10.707 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (3m) 9s ago 4m 60.7M - 514e6a882f6e e4b29328945a 2023-10-01T19:47:10.707 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (3m) 0s ago 3m 94.2M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T19:47:10.707 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (3m) 9s ago 3m 90.0M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T19:47:11.111 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions' 2023-10-01T19:47:11.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:47:11.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:47:11.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:47:11.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:11.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:47:11.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:47:11.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:47:11.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:47:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:47:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:47:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:12.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:12 smithi057 ceph-mon[114331]: from='client.24844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:47:12.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:12 smithi057 ceph-mon[114331]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:12.443 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:12 smithi057 ceph-mon[108286]: from='client.24844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:47:12.443 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:12 smithi057 ceph-mon[108286]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:12.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:12 smithi087 ceph-mon[109386]: from='client.24844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:47:12.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:12 smithi087 ceph-mon[109386]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:12.882 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T19:47:12.883 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2023-10-01T19:47:12.883 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2023-10-01T19:47:12.883 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:47:12.883 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2023-10-01T19:47:12.883 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T19:47:12.883 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:47:12.883 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2023-10-01T19:47:12.883 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2023-10-01T19:47:12.883 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:47:12.883 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": {}, 2023-10-01T19:47:12.883 INFO:teuthology.orchestra.run.smithi057.stdout: "rgw": { 2023-10-01T19:47:12.883 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T19:47:12.884 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:47:12.884 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2023-10-01T19:47:12.884 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 15 2023-10-01T19:47:12.884 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:47:12.884 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T19:47:13.271 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph -s' 2023-10-01T19:47:13.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:13 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1437615644' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T19:47:13.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:13 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1437615644' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T19:47:13.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:13 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1437615644' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T19:47:14.368 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:14 smithi057 ceph-mon[108286]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:14.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:14 smithi057 ceph-mon[114331]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:14.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:14 smithi087 ceph-mon[109386]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:14.820 INFO:teuthology.orchestra.run.smithi057.stdout: cluster: 2023-10-01T19:47:14.820 INFO:teuthology.orchestra.run.smithi057.stdout: id: 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:47:14.821 INFO:teuthology.orchestra.run.smithi057.stdout: health: HEALTH_OK 2023-10-01T19:47:14.821 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:47:14.821 INFO:teuthology.orchestra.run.smithi057.stdout: services: 2023-10-01T19:47:14.821 INFO:teuthology.orchestra.run.smithi057.stdout: mon: 3 daemons, quorum a,c,b (age 7m) 2023-10-01T19:47:14.821 INFO:teuthology.orchestra.run.smithi057.stdout: mgr: y(active, since 4m), standbys: x 2023-10-01T19:47:14.821 INFO:teuthology.orchestra.run.smithi057.stdout: osd: 8 osds: 8 up (since 4m), 8 in (since 4m) 2023-10-01T19:47:14.821 INFO:teuthology.orchestra.run.smithi057.stdout: rgw: 2 daemons active (2 hosts, 1 zones) 2023-10-01T19:47:14.821 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:47:14.821 INFO:teuthology.orchestra.run.smithi057.stdout: data: 2023-10-01T19:47:14.821 INFO:teuthology.orchestra.run.smithi057.stdout: pools: 6 pools, 161 pgs 2023-10-01T19:47:14.821 INFO:teuthology.orchestra.run.smithi057.stdout: objects: 209 objects, 585 KiB 2023-10-01T19:47:14.822 INFO:teuthology.orchestra.run.smithi057.stdout: usage: 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:47:14.822 INFO:teuthology.orchestra.run.smithi057.stdout: pgs: 161 active+clean 2023-10-01T19:47:14.822 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:47:14.822 INFO:teuthology.orchestra.run.smithi057.stdout: io: 2023-10-01T19:47:14.822 INFO:teuthology.orchestra.run.smithi057.stdout: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2023-10-01T19:47:14.822 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:47:15.228 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch ls' 2023-10-01T19:47:15.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:15 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2018072657' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T19:47:15.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:15 smithi057 ceph-mon[114331]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:15.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:15 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2018072657' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T19:47:15.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:15 smithi057 ceph-mon[108286]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:16.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:15 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2018072657' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T19:47:16.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:15 smithi087 ceph-mon[109386]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:16.294 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:47:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:47:16] "GET /metrics HTTP/1.1" 200 207960 "" "Prometheus/2.33.4" 2023-10-01T19:47:16.699 INFO:teuthology.orchestra.run.smithi057.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-01T19:47:16.699 INFO:teuthology.orchestra.run.smithi057.stdout:agent 2/2 15s ago 4m * 2023-10-01T19:47:16.699 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager ?:9093,9094 1/1 6s ago 4m smithi057=a;count:1 2023-10-01T19:47:16.700 INFO:teuthology.orchestra.run.smithi057.stdout:grafana ?:3000 1/1 15s ago 4m smithi087=a;count:1 2023-10-01T19:47:16.700 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo 1/1 6s ago 3m count:1 2023-10-01T19:47:16.700 INFO:teuthology.orchestra.run.smithi057.stdout:mgr 2/2 15s ago 7m smithi057=y;smithi087=x;count:2 2023-10-01T19:47:16.700 INFO:teuthology.orchestra.run.smithi057.stdout:mon 3/3 15s ago 7m smithi057:172.21.15.57=a;smithi057:[v2:172.21.15.57:3301,v1:172.21.15.57:6790]=c;smithi087:172.21.15.87=b;count:3 2023-10-01T19:47:16.700 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter ?:9100 2/2 15s ago 4m smithi057=a;smithi087=b;count:2 2023-10-01T19:47:16.700 INFO:teuthology.orchestra.run.smithi057.stdout:osd 8 15s ago - 2023-10-01T19:47:16.700 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus ?:9095 1/1 15s ago 4m smithi087=a;count:1 2023-10-01T19:47:16.700 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo ?:8000 2/2 15s ago 3m count:2 2023-10-01T19:47:17.083 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch daemon redeploy "mgr.$(ceph mgr dump -f json | jq .standbys | jq .[] | jq -r .name)" --image quay.ceph.io/ceph-ci/ceph:$sha1' 2023-10-01T19:47:18.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:18 smithi057 ceph-mon[108286]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:47:18.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:18 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:18.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:18 smithi057 ceph-mon[108286]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:18.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:18 smithi057 ceph-mon[114331]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:47:18.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:18 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:18.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:18 smithi057 ceph-mon[114331]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:18 smithi087 ceph-mon[109386]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:47:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:18 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:18 smithi087 ceph-mon[109386]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:18.902 INFO:teuthology.orchestra.run.smithi057.stdout:Scheduled to redeploy mgr.x on host 'smithi087' 2023-10-01T19:47:19.308 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch ps --refresh' 2023-10-01T19:47:19.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:19 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3621735737' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-01T19:47:19.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:19 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:19.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:19 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:47:19.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:19 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3621735737' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-01T19:47:19.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:19 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:19.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:19 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:47:19.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:19 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3621735737' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-01T19:47:19.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:19 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:19.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:19 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:47:20.194 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:47:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:47:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:47:20.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:20 smithi057 ceph-mon[108286]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "orch daemon redeploy", "name": "mgr.x", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:47:20.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:20 smithi057 ceph-mon[108286]: Schedule redeploy daemon mgr.x 2023-10-01T19:47:20.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:20 smithi057 ceph-mon[108286]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:20.394 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:20 smithi057 ceph-mon[114331]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "orch daemon redeploy", "name": "mgr.x", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:47:20.394 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:20 smithi057 ceph-mon[114331]: Schedule redeploy daemon mgr.x 2023-10-01T19:47:20.395 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:20 smithi057 ceph-mon[114331]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:20.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:20 smithi087 ceph-mon[109386]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "orch daemon redeploy", "name": "mgr.x", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:47:20.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:20 smithi087 ceph-mon[109386]: Schedule redeploy daemon mgr.x 2023-10-01T19:47:20.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:20 smithi087 ceph-mon[109386]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:20.830 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T19:47:20.830 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 10s ago 7m - - 2023-10-01T19:47:20.830 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 19s ago 7m - - 2023-10-01T19:47:20.830 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (3m) 10s ago 4m 29.4M - ba2b418f427c 1b7e399a166e 2023-10-01T19:47:20.831 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (3m) 19s ago 3m 60.9M - 8.3.5 dad864ee21e9 19c2251f4393 2023-10-01T19:47:20.831 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (3m) 10s ago 3m 56.5M - 3.5 e1d6a67b021e df9831dacebb 2023-10-01T19:47:20.831 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443 running (7m) 19s ago 7m 420M - 17.2.0 e1d6a67b021e d46b745be5ba 2023-10-01T19:47:20.831 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:9283 running (8m) 10s ago 8m 484M - 17.2.0 e1d6a67b021e d546d6fb718a 2023-10-01T19:47:20.831 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (8m) 10s ago 8m 97.2M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T19:47:20.831 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (7m) 19s ago 7m 61.7M 2048M 17.2.0 e1d6a67b021e 4734e639b775 2023-10-01T19:47:20.832 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (7m) 10s ago 7m 59.3M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T19:47:20.832 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (4m) 10s ago 4m 23.7M - 1dbe0e931976 efd276ccf21b 2023-10-01T19:47:20.832 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (4m) 19s ago 4m 20.2M - 1dbe0e931976 6d094569ea47 2023-10-01T19:47:20.832 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (6m) 10s ago 6m 87.7M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T19:47:20.832 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (6m) 10s ago 6m 93.1M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T19:47:20.832 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (6m) 10s ago 6m 73.1M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T19:47:20.833 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (6m) 10s ago 6m 89.9M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T19:47:20.833 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (5m) 19s ago 5m 87.4M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T19:47:20.833 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (5m) 19s ago 5m 86.5M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T19:47:20.833 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (5m) 19s ago 5m 73.3M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T19:47:20.833 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (4m) 19s ago 4m 86.9M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T19:47:20.833 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (3m) 19s ago 4m 60.7M - 514e6a882f6e e4b29328945a 2023-10-01T19:47:20.833 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (3m) 10s ago 3m 94.2M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T19:47:20.834 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (3m) 19s ago 3m 90.0M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T19:47:21.251 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'sleep 180' 2023-10-01T19:47:21.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:47:21 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:47:21] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:47:22.393 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:22 smithi057 ceph-mon[108286]: from='client.24880 -' entity='client.admin' cmd=[{"prefix": "orch ps", "refresh": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:47:22.403 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:22 smithi057 ceph-mon[108286]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:22.403 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:22 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:22.403 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:22 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:22.403 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:22 smithi057 ceph-mon[114331]: from='client.24880 -' entity='client.admin' cmd=[{"prefix": "orch ps", "refresh": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:47:22.405 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:22 smithi057 ceph-mon[114331]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:22.405 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:22 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:22.405 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:22 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:22.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:22 smithi087 ceph-mon[109386]: from='client.24880 -' entity='client.admin' cmd=[{"prefix": "orch ps", "refresh": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:47:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:22 smithi087 ceph-mon[109386]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:22 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:22 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:24.486 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:24 smithi087 ceph-mon[109386]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:24.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:24 smithi057 ceph-mon[108286]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:24.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:24 smithi057 ceph-mon[114331]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:26.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:25 smithi057 ceph-mon[108286]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:26.059 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:25 smithi057 ceph-mon[114331]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:26.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:25 smithi087 ceph-mon[109386]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:26.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:47:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:47:26] "GET /metrics HTTP/1.1" 200 207960 "" "Prometheus/2.33.4" 2023-10-01T19:47:28.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:28 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:28.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:28 smithi087 ceph-mon[109386]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:28 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:28 smithi057 ceph-mon[108286]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:28 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:28 smithi057 ceph-mon[114331]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:30.182 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:47:29 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:47:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:47:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:30 smithi087 ceph-mon[109386]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:30 smithi057 ceph-mon[108286]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:30.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:47:30 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:47:30] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:47:30.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:30 smithi057 ceph-mon[114331]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:31.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:31 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:31.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:31 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:31.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:31 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:31.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:31 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:31.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:31 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:31.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:31 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:32.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:32 smithi087 ceph-mon[109386]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:32.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:32 smithi057 ceph-mon[114331]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:32.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:32 smithi057 ceph-mon[108286]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:34.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:34 smithi087 ceph-mon[109386]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:34.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:34 smithi057 ceph-mon[108286]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:34.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:34 smithi057 ceph-mon[114331]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:36.059 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:35 smithi057 ceph-mon[108286]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:36.059 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:35 smithi057 ceph-mon[114331]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:36.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:35 smithi087 ceph-mon[109386]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:36.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:47:36 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:47:36] "GET /metrics HTTP/1.1" 200 207959 "" "Prometheus/2.33.4" 2023-10-01T19:47:38.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:38 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:38.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:38 smithi087 ceph-mon[109386]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:38.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:38 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:38.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:38 smithi057 ceph-mon[108286]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:38.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:38 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:38.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:38 smithi057 ceph-mon[114331]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:40.185 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:47:39 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:47:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:47:40.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:40 smithi087 ceph-mon[109386]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:40.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:40 smithi057 ceph-mon[108286]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:40.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:40 smithi057 ceph-mon[114331]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:42.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:47:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:47:41] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:47:42.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:42 smithi087 ceph-mon[109386]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:42.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:42 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:42.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:42 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:42.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:42 smithi057 ceph-mon[108286]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:42.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:42 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:42.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:42 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:42 smithi057 ceph-mon[114331]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:42 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:42 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:44.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:44 smithi087 ceph-mon[109386]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:44.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:44 smithi057 ceph-mon[108286]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:44.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:44 smithi057 ceph-mon[114331]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:45.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:47:45.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:47:45.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:45 smithi087 ceph-mon[109386]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:46.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:47:46.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:47:46.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:45 smithi057 ceph-mon[108286]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:46.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:47:46.059 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:47:46.059 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:45 smithi057 ceph-mon[114331]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:46.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:47:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:47:46] "GET /metrics HTTP/1.1" 200 207948 "" "Prometheus/2.33.4" 2023-10-01T19:47:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:48 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:48 smithi087 ceph-mon[109386]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:48.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:48 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:48.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:48 smithi057 ceph-mon[108286]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:48.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:48 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:48.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:48 smithi057 ceph-mon[114331]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:50.197 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:47:49 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:47:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:47:50.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:50 smithi057 ceph-mon[108286]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:50.477 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:50 smithi057 ceph-mon[114331]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:50.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:50 smithi087 ceph-mon[109386]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:50.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:47:50 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:47:50] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:47:51.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:51 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:51.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:51 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:51.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:51 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:51.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:51 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:51.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:51 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:51.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:51 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:47:52.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:52 smithi087 ceph-mon[109386]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:52.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:52 smithi057 ceph-mon[108286]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:52.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:52 smithi057 ceph-mon[114331]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:54.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:54 smithi087 ceph-mon[109386]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:54.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:54 smithi057 ceph-mon[108286]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:54.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:54 smithi057 ceph-mon[114331]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:56.057 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:55 smithi057 ceph-mon[108286]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:56.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:55 smithi057 ceph-mon[114331]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:56.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:55 smithi087 ceph-mon[109386]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:47:56.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:47:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:47:56] "GET /metrics HTTP/1.1" 200 207948 "" "Prometheus/2.33.4" 2023-10-01T19:47:58.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:58 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:58.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:47:58 smithi057 ceph-mon[108286]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:58.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:58 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:58.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:47:58 smithi057 ceph-mon[114331]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:47:58.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:58 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:47:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:47:58 smithi087 ceph-mon[109386]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:00.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:47:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:47:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:48:00.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[114331]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:00.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:48:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:48:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:48:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T19:48:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:48:00.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[108286]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:00.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:00.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:48:00.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:48:00.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:00.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:48:00.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T19:48:00.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:00 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:48:00.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:00 smithi087 ceph-mon[109386]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:00 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:00 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:48:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:00 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:48:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:00 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:00 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:48:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:00 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T19:48:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:00 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:48:01.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:01 smithi087 ceph-mon[109386]: Deploying daemon mgr.x on smithi087 2023-10-01T19:48:01.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:01 smithi057 ceph-mon[108286]: Deploying daemon mgr.x on smithi087 2023-10-01T19:48:01.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:01 smithi057 ceph-mon[114331]: Deploying daemon mgr.x on smithi087 2023-10-01T19:48:02.221 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:01 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:48:01] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:48:02.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:02 smithi087 ceph-mon[109386]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:02.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:02 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:02.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:02 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:02.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:02 smithi057 ceph-mon[108286]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:02.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:02 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:02.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:02 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:02.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:02 smithi057 ceph-mon[114331]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:02.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:02 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:02.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:02 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:03.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:03.509Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:48:03.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:03.509Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:48:04.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:04 smithi087 ceph-mon[109386]: pgmap v204: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:04.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:04 smithi057 ceph-mon[108286]: pgmap v204: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:04.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:04 smithi057 ceph-mon[114331]: pgmap v204: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:06.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:05 smithi057 ceph-mon[108286]: pgmap v205: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:06.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:05 smithi057 ceph-mon[114331]: pgmap v205: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:06.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:05 smithi087 ceph-mon[109386]: pgmap v205: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:06.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:06 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:48:06] "GET /metrics HTTP/1.1" 200 207957 "" "Prometheus/2.33.4" 2023-10-01T19:48:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:08 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:08 smithi087 ceph-mon[109386]: pgmap v206: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:08.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:08 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:08.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:08 smithi057 ceph-mon[108286]: pgmap v206: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:08.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:08 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:08.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:08 smithi057 ceph-mon[114331]: pgmap v206: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:10.203 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:09 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:48:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:48:10.610 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:10 smithi057 ceph-mon[108286]: pgmap v207: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:10.611 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:10 smithi057 ceph-mon[114331]: pgmap v207: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:10.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:10 smithi087 ceph-mon[109386]: pgmap v207: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:10.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:10 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:48:10] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:48:11.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:11.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:11 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:11.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:11.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:11 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:11.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:11.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:11 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:12.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:12 smithi087 ceph-mon[109386]: pgmap v208: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:12.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:12 smithi057 ceph-mon[108286]: pgmap v208: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:12.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:12 smithi057 ceph-mon[114331]: pgmap v208: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:13.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:48:13.508Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:48:13.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:13.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:48:13.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:13.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:48:14.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:14 smithi057 ceph-mon[108286]: pgmap v209: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-10-01T19:48:14.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:14 smithi057 ceph-mon[114331]: pgmap v209: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-10-01T19:48:15.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:14 smithi087 ceph-mon[109386]: pgmap v209: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-10-01T19:48:16.057 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:15 smithi057 ceph-mon[108286]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:16.057 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:15 smithi057 ceph-mon[114331]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:16.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:15 smithi087 ceph-mon[109386]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:16.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:48:16] "GET /metrics HTTP/1.1" 200 207966 "" "Prometheus/2.33.4" 2023-10-01T19:48:18.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:18 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:18 smithi087 ceph-mon[109386]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:18.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:18 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:18.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:18 smithi057 ceph-mon[108286]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:18.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:18 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:18.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:18 smithi057 ceph-mon[114331]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:20.200 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[111169]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:48:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:48:20.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:20 smithi087 ceph-mon[109386]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:20.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:20 smithi057 ceph-mon[108286]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:20.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:20 smithi057 ceph-mon[114331]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:22.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:22 smithi087 ceph-mon[109386]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:22 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:22 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:22.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:48:22] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:48:22.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:22 smithi057 ceph-mon[108286]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:22.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:22 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:22.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:22 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:22.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:22 smithi057 ceph-mon[114331]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:22.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:22 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:22.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:22 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:23.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:48:23.509Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:48:23.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:23.511Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:48:23.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:23.511Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:48:24.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:24 smithi087 ceph-mon[109386]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:24.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:24 smithi057 ceph-mon[108286]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:24.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:24 smithi057 ceph-mon[114331]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:26.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:25 smithi057 ceph-mon[108286]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:26.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:25 smithi057 ceph-mon[114331]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:26.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:25 smithi087 ceph-mon[109386]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:26.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:48:26] "GET /metrics HTTP/1.1" 200 207966 "" "Prometheus/2.33.4" 2023-10-01T19:48:28.488 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:28 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:28.694 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:28 smithi087 ceph-mon[109386]: pgmap v216: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:28.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:28 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:28.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:28 smithi057 ceph-mon[108286]: pgmap v216: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:28.697 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:28 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:28.697 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:28 smithi057 ceph-mon[114331]: pgmap v216: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:29.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:28 smithi087 systemd[1]: Stopping Ceph mgr.x for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:48:29.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:28 smithi087 bash[136907]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr.x" found: no such container 2023-10-01T19:48:29.436 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:29 smithi087 bash[136919]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x 2023-10-01T19:48:29.436 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:29 smithi087 bash[136956]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr.x" found: no such container 2023-10-01T19:48:29.436 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:29 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.x.service: Main process exited, code=exited, status=143/n/a 2023-10-01T19:48:29.436 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:29 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.x.service: Failed with result 'exit-code'. 2023-10-01T19:48:29.436 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:29 smithi087 systemd[1]: Stopped Ceph mgr.x for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:48:29.436 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:29 smithi087 systemd[1]: Starting Ceph mgr.x for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:48:30.114 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:29 smithi087 bash[137066]: f7b2c168ac86bb021efe0a92084283387027abe029a0375fd1d0a08fcfb81a2e 2023-10-01T19:48:30.114 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:29 smithi087 systemd[1]: Started Ceph mgr.x for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:48:30.114 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:29 smithi087 ceph-mgr[137104]: -- 172.21.15.87:0/1565797926 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558b675ed1e0 con 0x558b675c2800 2023-10-01T19:48:30.114 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:29 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:29.927+0000 7f932c678200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T19:48:30.115 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:30.105+0000 7f932c678200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T19:48:30.205 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:30.154Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=6 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T19:48:30.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:30 smithi087 ceph-mon[109386]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:48:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:48:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:48:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[108286]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:48:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:48:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:48:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:30.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[114331]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:48:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:48:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:48:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:30.841 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:30.460+0000 7f932c678200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T19:48:31.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:31 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:31.003+0000 7f932c678200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T19:48:31.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:30 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:48:30] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:48:31.632 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:31 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:31.234+0000 7f932c678200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-01T19:48:32.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:31 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:31.654+0000 7f932c678200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T19:48:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:31 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:31 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:31 smithi087 ceph-mon[109386]: pgmap v218: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:32.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:31 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:32.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:31 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:32.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:31 smithi057 ceph-mon[108286]: pgmap v218: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:32.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:31 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:32.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:31 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:32.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:31 smithi057 ceph-mon[114331]: pgmap v218: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:32.632 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:32 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:32.376+0000 7f932c678200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T19:48:32.633 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:32 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:32.466+0000 7f932c678200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T19:48:33.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:33.071+0000 7f932c678200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T19:48:33.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:33.159+0000 7f932c678200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T19:48:33.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:33.250+0000 7f932c678200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T19:48:33.776 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:48:33.510Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:48:33.777 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:33.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T19:48:33.777 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:33.512Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:48:33.880 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:33.475+0000 7f932c678200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T19:48:33.880 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:33.568+0000 7f932c678200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T19:48:33.881 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:33.664+0000 7f932c678200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T19:48:34.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:33.880+0000 7f932c678200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T19:48:34.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:33.978+0000 7f932c678200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T19:48:34.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:34 smithi087 ceph-mon[109386]: pgmap v219: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:34.633 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:34.166+0000 7f932c678200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T19:48:34.633 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:34.254+0000 7f932c678200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T19:48:34.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:34 smithi057 ceph-mon[108286]: pgmap v219: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:34.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:34 smithi057 ceph-mon[114331]: pgmap v219: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:35.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:35.010+0000 7f932c678200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T19:48:35.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:35.212+0000 7f932c678200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T19:48:35.794 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:35.395+0000 7f932c678200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:48:36.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:35 smithi087 ceph-mon[109386]: pgmap v220: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:36.059 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:35 smithi057 ceph-mon[108286]: pgmap v220: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:36.059 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:35 smithi057 ceph-mon[114331]: pgmap v220: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:36.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:36.054+0000 7f932c678200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T19:48:36.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:36.246+0000 7f932c678200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:48:36.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:48:36] "GET /metrics HTTP/1.1" 200 207975 "" "Prometheus/2.33.4" 2023-10-01T19:48:36.802 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:36.446+0000 7f932c678200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T19:48:36.802 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:36.568+0000 7f932c678200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T19:48:36.802 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:48:36] ENGINE Bus STARTING 2023-10-01T19:48:36.802 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: CherryPy Checker: 2023-10-01T19:48:36.802 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: The Application mounted at '' has an empty config. 2023-10-01T19:48:36.802 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:48:36.802 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:48:36] ENGINE Serving on http://:::9283 2023-10-01T19:48:36.802 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:48:36] ENGINE Bus STARTED 2023-10-01T19:48:37.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-mon[108286]: Standby manager daemon x restarted 2023-10-01T19:48:37.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-mon[108286]: Standby manager daemon x started 2023-10-01T19:48:37.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T19:48:37.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:48:37.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T19:48:37.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:48:37.092 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-mon[114331]: Standby manager daemon x restarted 2023-10-01T19:48:37.092 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-mon[114331]: Standby manager daemon x started 2023-10-01T19:48:37.092 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T19:48:37.092 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:48:37.092 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T19:48:37.092 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:36 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:48:37.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-mon[109386]: Standby manager daemon x restarted 2023-10-01T19:48:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-mon[109386]: Standby manager daemon x started 2023-10-01T19:48:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T19:48:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:48:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T19:48:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:36 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:48:38.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:37 smithi087 ceph-mon[109386]: mgrmap e22: y(active, since 5m), standbys: x 2023-10-01T19:48:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:37 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:37 smithi087 ceph-mon[109386]: pgmap v221: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:38.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:37 smithi057 ceph-mon[114331]: mgrmap e22: y(active, since 5m), standbys: x 2023-10-01T19:48:38.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:37 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:38.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:37 smithi057 ceph-mon[114331]: pgmap v221: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:38.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:37 smithi057 ceph-mon[108286]: mgrmap e22: y(active, since 5m), standbys: x 2023-10-01T19:48:38.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:37 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:38.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:37 smithi057 ceph-mon[108286]: pgmap v221: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:38.671 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:38 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:38.262Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=5 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:48:40.204 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:39 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:48:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:48:40.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:40 smithi087 ceph-mon[109386]: pgmap v222: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:40.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:40 smithi057 ceph-mon[108286]: pgmap v222: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:40.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:40 smithi057 ceph-mon[114331]: pgmap v222: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:42.488 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:42 smithi087 ceph-mon[109386]: pgmap v223: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:42.488 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:42 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:42.488 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:42 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:42.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:42 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:48:42] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:48:42.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:42 smithi057 ceph-mon[108286]: pgmap v223: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:42.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:42 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:42.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:42 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:42 smithi057 ceph-mon[114331]: pgmap v223: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:42 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:42 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:43.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:48:43] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:48:43.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:48:43.510Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:48:43.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:43.512Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:48:43.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:43.513Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:48:44.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:44 smithi057 ceph-mon[108286]: pgmap v224: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:44.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:44 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:44.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:44 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:44.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:44 smithi057 ceph-mon[114331]: pgmap v224: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:44.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:44 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:44.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:44 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:45.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:44 smithi087 ceph-mon[109386]: pgmap v224: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:45.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:44 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:45.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:44 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:46.057 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:48:46.057 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:48:46.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:45 smithi057 ceph-mon[108286]: pgmap v225: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:46.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:48:46.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:48:46.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:45 smithi057 ceph-mon[114331]: pgmap v225: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:46.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:48:46.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:48:46.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:45 smithi087 ceph-mon[109386]: pgmap v225: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:46.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:48:46] "GET /metrics HTTP/1.1" 200 207970 "" "Prometheus/2.33.4" 2023-10-01T19:48:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:48 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:48 smithi087 ceph-mon[109386]: pgmap v226: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:48.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:48 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:48.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:48 smithi057 ceph-mon[114331]: pgmap v226: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:48.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:48 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:48.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:48 smithi057 ceph-mon[108286]: pgmap v226: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:50.207 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:49 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:48:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:48:50.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:50 smithi087 ceph-mon[109386]: pgmap v227: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:50.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:50 smithi057 ceph-mon[108286]: pgmap v227: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:50.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:50 smithi057 ceph-mon[114331]: pgmap v227: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:51.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:50 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:48:50] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:48:52.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:51 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:52.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:51 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:52.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:51 smithi087 ceph-mon[109386]: pgmap v228: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:52.167 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:51 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:52.168 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:51 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:52.168 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:51 smithi057 ceph-mon[108286]: pgmap v228: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:52.168 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:51 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:52.168 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:51 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:48:52.168 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:51 smithi057 ceph-mon[114331]: pgmap v228: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:53.671 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:48:53.511Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=7 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:48:53.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:53.513Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:48:53.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:48:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:48:53.513Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:48:54.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:54 smithi087 ceph-mon[109386]: pgmap v229: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:54.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:54 smithi057 ceph-mon[108286]: pgmap v229: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:54.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:54 smithi057 ceph-mon[114331]: pgmap v229: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:56.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:55 smithi057 ceph-mon[108286]: pgmap v230: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:56.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:55 smithi057 ceph-mon[114331]: pgmap v230: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:56.131 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:55 smithi087 ceph-mon[109386]: pgmap v230: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:48:56.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:48:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:48:56] "GET /metrics HTTP/1.1" 200 207970 "" "Prometheus/2.33.4" 2023-10-01T19:48:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:58 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:48:58 smithi087 ceph-mon[109386]: pgmap v231: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:58.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:58 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:58.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:48:58 smithi057 ceph-mon[108286]: pgmap v231: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:48:58.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:58 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:48:58.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:48:58 smithi057 ceph-mon[114331]: pgmap v231: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:00.230 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:48:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:48:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:49:00.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:00 smithi087 ceph-mon[109386]: pgmap v232: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:00.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:00 smithi057 ceph-mon[108286]: pgmap v232: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:00.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:00 smithi057 ceph-mon[114331]: pgmap v232: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:02.625 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:02 smithi087 ceph-mon[109386]: pgmap v233: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:02.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:02 smithi057 ceph-mon[108286]: pgmap v233: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:02.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:02 smithi057 ceph-mon[114331]: pgmap v233: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:03.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:49:02 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:49:02] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:49:03.891 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:49:03.511Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:49:03.891 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:49:03.513Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:49:03.891 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:49:03.514Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:49:04.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:03 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:04.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:03 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:04.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:03 smithi057 ceph-mon[108286]: pgmap v234: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:04.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:03 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:04.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:03 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:04.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:03 smithi057 ceph-mon[114331]: pgmap v234: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:04.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:03 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:04.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:03 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:04.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:03 smithi087 ceph-mon[109386]: pgmap v234: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:06.057 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:05 smithi057 ceph-mon[108286]: pgmap v235: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:06.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:05 smithi057 ceph-mon[114331]: pgmap v235: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:06.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:05 smithi087 ceph-mon[109386]: pgmap v235: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:06.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:49:06 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:49:06] "GET /metrics HTTP/1.1" 200 207963 "" "Prometheus/2.33.4" 2023-10-01T19:49:08.529 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:08 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:08.529 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:08 smithi087 ceph-mon[109386]: pgmap v236: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:08.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:08 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:08.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:08 smithi057 ceph-mon[108286]: pgmap v236: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:08.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:08 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:08.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:08 smithi057 ceph-mon[114331]: pgmap v236: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:10.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:49:09 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:49:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:49:10.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:10 smithi087 ceph-mon[109386]: pgmap v237: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:10.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:10 smithi057 ceph-mon[108286]: pgmap v237: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:10.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:10 smithi057 ceph-mon[114331]: pgmap v237: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:11.422 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:49:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:49:11] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:49:12.264 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:12 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:12.264 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:12 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:12.265 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:12 smithi057 ceph-mon[108286]: pgmap v238: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:12.265 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:12 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:12.265 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:12 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:12.265 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:12 smithi057 ceph-mon[114331]: pgmap v238: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:12.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:12 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:12.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:12 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:12.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:12 smithi087 ceph-mon[109386]: pgmap v238: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:13.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:49:13.512Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:49:13.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:49:13.514Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:49:13.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:49:13.515Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:49:14.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:14 smithi087 ceph-mon[109386]: pgmap v239: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:14.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:14 smithi057 ceph-mon[108286]: pgmap v239: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:14.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:14 smithi057 ceph-mon[114331]: pgmap v239: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:16.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:15 smithi087 ceph-mon[109386]: pgmap v240: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:16.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:15 smithi057 ceph-mon[108286]: pgmap v240: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:15 smithi057 ceph-mon[114331]: pgmap v240: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:16.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:49:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:49:16] "GET /metrics HTTP/1.1" 200 207966 "" "Prometheus/2.33.4" 2023-10-01T19:49:18.209 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:18 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:18.209 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:18 smithi057 ceph-mon[108286]: pgmap v241: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:18 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:18 smithi087 ceph-mon[109386]: pgmap v241: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:18.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:18 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:18.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:18 smithi057 ceph-mon[114331]: pgmap v241: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:20.229 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:49:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:49:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:49:20.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:20 smithi087 ceph-mon[109386]: pgmap v242: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:20.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:20 smithi057 ceph-mon[108286]: pgmap v242: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:20.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:20 smithi057 ceph-mon[114331]: pgmap v242: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:22 smithi087 ceph-mon[109386]: pgmap v243: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:22.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:22 smithi057 ceph-mon[108286]: pgmap v243: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:22.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:22 smithi057 ceph-mon[114331]: pgmap v243: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:23.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:49:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:49:23] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:49:23.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:49:23.513Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:49:23.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:49:23.515Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:49:23.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:49:23.515Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:49:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:24 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:24 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:24 smithi087 ceph-mon[109386]: pgmap v244: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:24.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:24 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:24.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:24 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:24.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:24 smithi057 ceph-mon[108286]: pgmap v244: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:24.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:24 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:24.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:24 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:24.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:24 smithi057 ceph-mon[114331]: pgmap v244: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:26.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:25 smithi057 ceph-mon[108286]: pgmap v245: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:26.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:25 smithi057 ceph-mon[114331]: pgmap v245: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:26.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:25 smithi087 ceph-mon[109386]: pgmap v245: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:26.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:49:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:49:26] "GET /metrics HTTP/1.1" 200 207966 "" "Prometheus/2.33.4" 2023-10-01T19:49:28.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:28 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:28.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:28 smithi087 ceph-mon[109386]: pgmap v246: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:28 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:28 smithi057 ceph-mon[108286]: pgmap v246: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:28.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:28 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:28 smithi057 ceph-mon[114331]: pgmap v246: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:30.222 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:49:29 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:49:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:49:30.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:30 smithi087 ceph-mon[109386]: pgmap v247: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:49:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:49:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:49:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:30.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:30 smithi057 ceph-mon[108286]: pgmap v247: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:49:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:49:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:49:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:30.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:30 smithi057 ceph-mon[114331]: pgmap v247: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:30.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:49:30.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:49:30.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:49:30.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:30.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:31.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:49:31 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:49:31] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:49:32.328 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:32 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:32.328 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:32 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:32.328 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:32 smithi057 ceph-mon[108286]: pgmap v248: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:32.329 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:32 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:32.329 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:32 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:32.329 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:32 smithi057 ceph-mon[114331]: pgmap v248: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:32.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:32 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:32.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:32 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:32.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:32 smithi087 ceph-mon[109386]: pgmap v248: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:33.912 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:49:33.514Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:49:33.912 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:49:33.516Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:49:33.912 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:49:33.516Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:49:34.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:34 smithi087 ceph-mon[109386]: pgmap v249: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:34.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:34 smithi057 ceph-mon[108286]: pgmap v249: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:34.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:34 smithi057 ceph-mon[114331]: pgmap v249: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:36.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:35 smithi087 ceph-mon[109386]: pgmap v250: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:36.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:35 smithi057 ceph-mon[108286]: pgmap v250: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:36.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:49:36 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:49:36] "GET /metrics HTTP/1.1" 200 207967 "" "Prometheus/2.33.4" 2023-10-01T19:49:36.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:35 smithi057 ceph-mon[114331]: pgmap v250: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:38.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:38 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:38.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:38 smithi087 ceph-mon[109386]: pgmap v251: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:38.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:38 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:38.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:38 smithi057 ceph-mon[114331]: pgmap v251: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:38 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:38 smithi057 ceph-mon[108286]: pgmap v251: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:40.229 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:49:39 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:49:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:49:40.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:40 smithi087 ceph-mon[109386]: pgmap v252: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:40.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:40 smithi057 ceph-mon[108286]: pgmap v252: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:40.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:40 smithi057 ceph-mon[114331]: pgmap v252: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:42.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:42 smithi087 ceph-mon[109386]: pgmap v253: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:42.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:42 smithi057 ceph-mon[108286]: pgmap v253: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:42.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:42 smithi057 ceph-mon[114331]: pgmap v253: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:43.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:49:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:49:43] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:49:43.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:49:43.515Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:49:43.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:49:43.517Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:49:43.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:49:43.517Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:49:44.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:44 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:44 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:44 smithi057 ceph-mon[108286]: pgmap v254: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:44.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:44 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:44.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:44 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:44.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:44 smithi057 ceph-mon[114331]: pgmap v254: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:44.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:44 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:44.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:44 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:44.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:44 smithi087 ceph-mon[109386]: pgmap v254: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:45.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:49:45.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:45 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:49:45.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:49:45.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:45 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:49:45.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:49:45.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:45 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:49:46.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:46 smithi057 ceph-mon[108286]: pgmap v255: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:46.422 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:49:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:49:46] "GET /metrics HTTP/1.1" 200 207974 "" "Prometheus/2.33.4" 2023-10-01T19:49:46.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:46 smithi057 ceph-mon[114331]: pgmap v255: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:46.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:46 smithi087 ceph-mon[109386]: pgmap v255: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:48.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:48 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:48.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:48 smithi087 ceph-mon[109386]: pgmap v256: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:48.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:48 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:48.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:48 smithi057 ceph-mon[114331]: pgmap v256: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:48.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:48 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:48.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:48 smithi057 ceph-mon[108286]: pgmap v256: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:50.208 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:49:49 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:49:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:49:50.540 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:50 smithi087 ceph-mon[109386]: pgmap v257: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:50.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:50 smithi057 ceph-mon[108286]: pgmap v257: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:50.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:50 smithi057 ceph-mon[114331]: pgmap v257: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:51.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:49:51 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:49:51] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:49:52.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:52 smithi057 ceph-mon[108286]: pgmap v258: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:52.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:52.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:52 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:52.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:52 smithi057 ceph-mon[114331]: pgmap v258: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:52.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:52.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:52 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:52.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:52 smithi087 ceph-mon[109386]: pgmap v258: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:52.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:52.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:52 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:49:53.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:49:53.515Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:49:53.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:49:53.518Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:49:53.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:49:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:49:53.518Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:49:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:54 smithi087 ceph-mon[109386]: pgmap v259: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:54.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:54 smithi057 ceph-mon[108286]: pgmap v259: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:54.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:54 smithi057 ceph-mon[114331]: pgmap v259: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:56.058 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:55 smithi057 ceph-mon[108286]: pgmap v260: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:56.058 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:55 smithi057 ceph-mon[114331]: pgmap v260: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:56.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:55 smithi087 ceph-mon[109386]: pgmap v260: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:49:56.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:49:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:49:56] "GET /metrics HTTP/1.1" 200 207974 "" "Prometheus/2.33.4" 2023-10-01T19:49:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:58 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:49:58 smithi087 ceph-mon[109386]: pgmap v261: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:58.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:58 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:58.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:49:58 smithi057 ceph-mon[108286]: pgmap v261: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:49:58.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:58 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:49:58.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:49:58 smithi057 ceph-mon[114331]: pgmap v261: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:00.231 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:49:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:49:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:50:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:00 smithi087 ceph-mon[109386]: pgmap v262: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:00 smithi087 ceph-mon[109386]: overall HEALTH_OK 2023-10-01T19:50:00.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:00 smithi057 ceph-mon[108286]: pgmap v262: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:00.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:00 smithi057 ceph-mon[108286]: overall HEALTH_OK 2023-10-01T19:50:00.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:00 smithi057 ceph-mon[114331]: pgmap v262: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:00.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:00 smithi057 ceph-mon[114331]: overall HEALTH_OK 2023-10-01T19:50:02.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:02 smithi087 ceph-mon[109386]: pgmap v263: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:02.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:02 smithi057 ceph-mon[108286]: pgmap v263: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:02.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:02 smithi057 ceph-mon[114331]: pgmap v263: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:03.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:50:03] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:50:03.671 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:50:03.516Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:50:03.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:03.518Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:50:03.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:03 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:03.519Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:50:04.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:04 smithi087 ceph-mon[109386]: pgmap v264: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:04.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:04 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:04.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:04 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:04.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:04 smithi057 ceph-mon[108286]: pgmap v264: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:04.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:04 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:04.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:04 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:04.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:04 smithi057 ceph-mon[114331]: pgmap v264: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:04.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:04 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:04.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:04 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:06.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:05 smithi087 ceph-mon[109386]: pgmap v265: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:06.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:05 smithi057 ceph-mon[108286]: pgmap v265: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:06.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:06 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:50:06] "GET /metrics HTTP/1.1" 200 207970 "" "Prometheus/2.33.4" 2023-10-01T19:50:06.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:05 smithi057 ceph-mon[114331]: pgmap v265: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:08 smithi087 ceph-mon[109386]: pgmap v266: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:08 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:08.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:08 smithi057 ceph-mon[108286]: pgmap v266: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:08.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:08 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:08.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:08 smithi057 ceph-mon[114331]: pgmap v266: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:08.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:08 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:10.250 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:50:09 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:50:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:50:10.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:10 smithi087 ceph-mon[109386]: pgmap v267: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:10.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:10 smithi057 ceph-mon[108286]: pgmap v267: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:10.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:10 smithi057 ceph-mon[114331]: pgmap v267: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:11.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:50:11] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:50:12.663 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:12 smithi087 ceph-mon[109386]: pgmap v268: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:12.663 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:12 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:12.663 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:12 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:12 smithi057 ceph-mon[108286]: pgmap v268: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:12 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:12 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:12.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:12 smithi057 ceph-mon[114331]: pgmap v268: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:12.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:12 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:12.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:12 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:13.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:50:13.516Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:50:13.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:13.519Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:50:13.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:13.519Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:50:14.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:14 smithi057 ceph-mon[108286]: pgmap v269: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:14.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:14 smithi057 ceph-mon[114331]: pgmap v269: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:14.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:14 smithi087 ceph-mon[109386]: pgmap v269: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:16.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:15 smithi087 ceph-mon[109386]: pgmap v270: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:16.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:15 smithi057 ceph-mon[108286]: pgmap v270: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:16.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:50:16] "GET /metrics HTTP/1.1" 200 207969 "" "Prometheus/2.33.4" 2023-10-01T19:50:16.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:15 smithi057 ceph-mon[114331]: pgmap v270: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:18.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:18 smithi087 ceph-mon[109386]: pgmap v271: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:18 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:18.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:18 smithi057 ceph-mon[108286]: pgmap v271: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:18.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:18 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:18.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:18 smithi057 ceph-mon[114331]: pgmap v271: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:18.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:18 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:20.245 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:50:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:50:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:50:20.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:20 smithi087 ceph-mon[109386]: pgmap v272: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:20.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:20 smithi057 ceph-mon[108286]: pgmap v272: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:20.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:20 smithi057 ceph-mon[114331]: pgmap v272: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:22.613 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:22 smithi057 ceph-mon[108286]: pgmap v273: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:22.613 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:22 smithi057 ceph-mon[114331]: pgmap v273: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:22.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:22 smithi087 ceph-mon[109386]: pgmap v273: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:22.833 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch ps' 2023-10-01T19:50:23.517 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.87 - - [01/Oct/2023:19:50:23] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:50:23.881 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:50:23.517Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:50:23.881 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:23.519Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:50:23.881 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:23.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:50:24.274 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T19:50:24.274 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 12s ago 10m - - 2023-10-01T19:50:24.274 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 0s ago 10m - - 2023-10-01T19:50:24.274 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (6m) 12s ago 7m 29.2M - ba2b418f427c 1b7e399a166e 2023-10-01T19:50:24.274 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (6m) 0s ago 6m 61.1M - 8.3.5 dad864ee21e9 19c2251f4393 2023-10-01T19:50:24.274 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (6m) 12s ago 6m 57.2M - 3.5 e1d6a67b021e df9831dacebb 2023-10-01T19:50:24.274 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283 running (114s) 0s ago 10m 451M - 18.0.0-6505-g37d71fd3 bea6d6056140 f7b2c168ac86 2023-10-01T19:50:24.275 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:9283 running (11m) 12s ago 11m 492M - 17.2.0 e1d6a67b021e d546d6fb718a 2023-10-01T19:50:24.275 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (11m) 12s ago 11m 109M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T19:50:24.275 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (10m) 0s ago 10m 67.7M 2048M 17.2.0 e1d6a67b021e 4734e639b775 2023-10-01T19:50:24.275 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (10m) 12s ago 10m 62.9M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T19:50:24.275 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (7m) 12s ago 7m 25.4M - 1dbe0e931976 efd276ccf21b 2023-10-01T19:50:24.275 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (7m) 0s ago 7m 21.1M - 1dbe0e931976 6d094569ea47 2023-10-01T19:50:24.275 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (9m) 12s ago 9m 99.9M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T19:50:24.276 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (9m) 12s ago 9m 106M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T19:50:24.276 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (9m) 12s ago 9m 82.1M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T19:50:24.276 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (9m) 12s ago 9m 106M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T19:50:24.276 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (8m) 0s ago 8m 104M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T19:50:24.276 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (8m) 0s ago 8m 102M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T19:50:24.276 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (8m) 0s ago 8m 85.1M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T19:50:24.276 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (8m) 0s ago 8m 101M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T19:50:24.277 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (6m) 0s ago 7m 70.9M - 514e6a882f6e e4b29328945a 2023-10-01T19:50:24.277 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (6m) 12s ago 6m 95.2M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T19:50:24.277 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (6m) 0s ago 6m 91.0M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T19:50:24.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:24 smithi057 ceph-mon[108286]: pgmap v274: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:24.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:24 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:24.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:24 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:24.599 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:24 smithi057 ceph-mon[114331]: pgmap v274: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:24.600 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:24 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:24.600 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:24 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:24.690 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions' 2023-10-01T19:50:24.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:24 smithi087 ceph-mon[109386]: pgmap v274: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:24.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:24 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:24.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:24 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:25.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:25 smithi057 ceph-mon[108286]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:50:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:25 smithi057 ceph-mon[114331]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:50:25.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:25 smithi087 ceph-mon[109386]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:50:26.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:50:26] "GET /metrics HTTP/1.1" 200 207969 "" "Prometheus/2.33.4" 2023-10-01T19:50:26.296 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T19:50:26.296 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2023-10-01T19:50:26.296 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2023-10-01T19:50:26.296 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:50:26.296 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2023-10-01T19:50:26.296 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2023-10-01T19:50:26.296 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 1 2023-10-01T19:50:26.296 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:50:26.296 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2023-10-01T19:50:26.297 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2023-10-01T19:50:26.297 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:50:26.297 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": {}, 2023-10-01T19:50:26.297 INFO:teuthology.orchestra.run.smithi057.stdout: "rgw": { 2023-10-01T19:50:26.297 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T19:50:26.297 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:50:26.297 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2023-10-01T19:50:26.297 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 14, 2023-10-01T19:50:26.297 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 1 2023-10-01T19:50:26.297 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:50:26.297 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T19:50:26.641 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:26 smithi057 ceph-mon[108286]: pgmap v275: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:26.641 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:26 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1580760754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T19:50:26.641 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:26 smithi057 ceph-mon[114331]: pgmap v275: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:26.641 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:26 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1580760754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T19:50:26.714 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph -s' 2023-10-01T19:50:26.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:26 smithi087 ceph-mon[109386]: pgmap v275: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:26.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:26 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1580760754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T19:50:28.369 INFO:teuthology.orchestra.run.smithi057.stdout: cluster: 2023-10-01T19:50:28.369 INFO:teuthology.orchestra.run.smithi057.stdout: id: 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:50:28.370 INFO:teuthology.orchestra.run.smithi057.stdout: health: HEALTH_OK 2023-10-01T19:50:28.370 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:50:28.370 INFO:teuthology.orchestra.run.smithi057.stdout: services: 2023-10-01T19:50:28.370 INFO:teuthology.orchestra.run.smithi057.stdout: mon: 3 daemons, quorum a,c,b (age 10m) 2023-10-01T19:50:28.370 INFO:teuthology.orchestra.run.smithi057.stdout: mgr: y(active, since 7m), standbys: x 2023-10-01T19:50:28.370 INFO:teuthology.orchestra.run.smithi057.stdout: osd: 8 osds: 8 up (since 7m), 8 in (since 8m) 2023-10-01T19:50:28.370 INFO:teuthology.orchestra.run.smithi057.stdout: rgw: 2 daemons active (2 hosts, 1 zones) 2023-10-01T19:50:28.370 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:50:28.370 INFO:teuthology.orchestra.run.smithi057.stdout: data: 2023-10-01T19:50:28.370 INFO:teuthology.orchestra.run.smithi057.stdout: pools: 6 pools, 161 pgs 2023-10-01T19:50:28.370 INFO:teuthology.orchestra.run.smithi057.stdout: objects: 209 objects, 585 KiB 2023-10-01T19:50:28.370 INFO:teuthology.orchestra.run.smithi057.stdout: usage: 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:28.371 INFO:teuthology.orchestra.run.smithi057.stdout: pgs: 161 active+clean 2023-10-01T19:50:28.371 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:50:28.371 INFO:teuthology.orchestra.run.smithi057.stdout: io: 2023-10-01T19:50:28.371 INFO:teuthology.orchestra.run.smithi057.stdout: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2023-10-01T19:50:28.371 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:50:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:28 smithi057 ceph-mon[108286]: pgmap v276: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:28 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:28 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2584866253' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T19:50:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:28 smithi057 ceph-mon[114331]: pgmap v276: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:28 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:28 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2584866253' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T19:50:28.873 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph health detail' 2023-10-01T19:50:28.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:28 smithi087 ceph-mon[109386]: pgmap v276: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:28.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:28 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:28.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:28 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2584866253' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T19:50:30.242 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:50:29 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:50:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:50:30.494 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:30 smithi087 ceph-mon[109386]: pgmap v277: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:30.494 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:50:30.494 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:50:30.494 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:30 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:50:30.623 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2023-10-01T19:50:30.811 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:30 smithi057 ceph-mon[108286]: pgmap v277: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:30.811 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:50:30.811 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:50:30.811 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:30 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:50:30.811 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:30 smithi057 ceph-mon[114331]: pgmap v277: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:30.811 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:50:30.811 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:50:30.812 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:30 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:50:31.030 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 2'"'"'' 2023-10-01T19:50:31.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:31 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 172.21.15.57 - - [01/Oct/2023:19:50:31] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.6" 2023-10-01T19:50:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:31 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:31 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:31 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/4018814563' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T19:50:31.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:31 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:31.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:31 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:31.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:31 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/4018814563' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T19:50:31.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:31 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:31.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:31 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:31.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:31 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/4018814563' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T19:50:32.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:32 smithi087 ceph-mon[109386]: pgmap v278: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:32.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:32 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:32.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:32 smithi087 ceph-mon[109386]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:32.691 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T19:50:32.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:32 smithi057 ceph-mon[108286]: pgmap v278: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:32.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:32 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:32.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:32 smithi057 ceph-mon[108286]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:32.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:32 smithi057 ceph-mon[114331]: pgmap v278: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:32.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:32 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:32.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:32 smithi057 ceph-mon[114331]: from='mgr.24343 172.21.15.57:0/741188837' entity='mgr.y' 2023-10-01T19:50:33.170 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph mgr fail' 2023-10-01T19:50:33.610 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:33 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3226082392' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T19:50:33.611 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:33 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3226082392' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T19:50:33.611 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:50:33.518Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:50:33.611 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:33.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:50:33.611 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:33.521Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:50:33.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:33 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3226082392' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T19:50:34.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:34 smithi057 ceph-mon[108286]: pgmap v279: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:34.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:34 smithi057 ceph-mon[114331]: pgmap v279: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:34.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:34 smithi087 ceph-mon[109386]: pgmap v279: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:36.072 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:35 smithi087 ceph-mon[109386]: pgmap v280: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:36.072 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:35 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/759752318' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T19:50:36.072 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:35 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T19:50:36.072 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:35 smithi087 ceph-mon[109386]: osdmap e91: 8 total, 8 up, 8 in 2023-10-01T19:50:36.073 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:50:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:50:35] ENGINE Bus STOPPING 2023-10-01T19:50:36.073 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:50:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:50:35] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-10-01T19:50:36.073 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:50:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:50:35] ENGINE Bus STOPPED 2023-10-01T19:50:36.073 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:50:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:50:35] ENGINE Bus STARTING 2023-10-01T19:50:36.089 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:35.999Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=4 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T19:50:36.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-mon[108286]: pgmap v280: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:36.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/759752318' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T19:50:36.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T19:50:36.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-mon[108286]: osdmap e91: 8 total, 8 up, 8 in 2023-10-01T19:50:36.090 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-mon[114331]: pgmap v280: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:50:36.090 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/759752318' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T19:50:36.090 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T19:50:36.090 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-mon[114331]: osdmap e91: 8 total, 8 up, 8 in 2023-10-01T19:50:36.091 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:35.830+0000 7f9d3c896700 -1 mgr handle_mgr_map I was active but no longer am 2023-10-01T19:50:36.091 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ignoring --setuser ceph since I am not root 2023-10-01T19:50:36.091 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ignoring --setgroup ceph since I am not root 2023-10-01T19:50:36.091 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:35 smithi057 ceph-mgr[108556]: -- 172.21.15.57:0/731190644 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a709306340 con 0x55a7092df000 2023-10-01T19:50:36.091 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:36.036+0000 7f96998e3000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T19:50:36.225 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'sleep 180' 2023-10-01T19:50:36.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:50:36] ENGINE Serving on http://:::9283 2023-10-01T19:50:36.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:50:36] ENGINE Bus STARTED 2023-10-01T19:50:36.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:36.233+0000 7f96998e3000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T19:50:36.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:36.625+0000 7f96998e3000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T19:50:36.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr fail"}]': finished 2023-10-01T19:50:36.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: mgrmap e23: x(active, starting, since 0.587378s) 2023-10-01T19:50:36.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:50:36.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:50:36.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:50:36.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T19:50:36.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:50:36.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:50:36.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:50:36.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:50:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:50:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:50:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:50:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:50:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:50:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:50:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:50:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: Manager daemon x is now available 2023-10-01T19:50:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: Queued rgw.foo for migration 2023-10-01T19:50:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'foo', 'service_name': 'rgw.foo', 'service_type': 'rgw', 'spec': {'rgw_frontend_port': 8000, 'rgw_realm': 'r', 'rgw_zone': 'z'}} 2023-10-01T19:50:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:36.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:36.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:50:36.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:50:36.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:50:36.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:50:36.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:50:36.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr fail"}]': finished 2023-10-01T19:50:36.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: mgrmap e23: x(active, starting, since 0.587378s) 2023-10-01T19:50:36.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:50:36.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:50:36.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:50:36.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T19:50:36.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:50:36.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:50:36.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:50:36.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:50:36.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:50:36.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:50:36.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:50:36.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:50:36.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:50:36.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:50:36.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:50:36.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: Manager daemon x is now available 2023-10-01T19:50:36.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:36.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: Queued rgw.foo for migration 2023-10-01T19:50:36.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'foo', 'service_name': 'rgw.foo', 'service_type': 'rgw', 'spec': {'rgw_frontend_port': 8000, 'rgw_realm': 'r', 'rgw_zone': 'z'}} 2023-10-01T19:50:36.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:36.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:36.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:50:36.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:50:36.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:50:36.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:50:36.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:50:36.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:50:36.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:50:36.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:50:36.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:50:36.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:50:36.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:50:36.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:50:36.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:50:36.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-01T19:50:36.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-01T19:50:36.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:36.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:36.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T19:50:36.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T19:50:36.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: [01/Oct/2023:19:50:36] ENGINE Bus STARTING 2023-10-01T19:50:36.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: [01/Oct/2023:19:50:36] ENGINE Serving on http://172.21.15.87:8765 2023-10-01T19:50:36.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:50:36.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:50:36.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:50:36.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:50:36.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:50:36.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:50:36.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:50:36.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:50:36.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:50:36.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:50:36.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-01T19:50:36.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-01T19:50:36.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:36.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:36.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T19:50:36.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T19:50:36.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: [01/Oct/2023:19:50:36] ENGINE Bus STARTING 2023-10-01T19:50:36.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: [01/Oct/2023:19:50:36] ENGINE Serving on http://172.21.15.87:8765 2023-10-01T19:50:36.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:50:36.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:36 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:50:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr fail"}]': finished 2023-10-01T19:50:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: mgrmap e23: x(active, starting, since 0.587378s) 2023-10-01T19:50:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:50:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:50:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:50:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T19:50:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:50:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:50:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:50:37.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:50:37.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:50:37.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:50:37.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:50:37.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:50:37.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:50:37.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:50:37.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:50:37.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: Manager daemon x is now available 2023-10-01T19:50:37.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:37.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: Queued rgw.foo for migration 2023-10-01T19:50:37.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'foo', 'service_name': 'rgw.foo', 'service_type': 'rgw', 'spec': {'rgw_frontend_port': 8000, 'rgw_realm': 'r', 'rgw_zone': 'z'}} 2023-10-01T19:50:37.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:37.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:37.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:50:37.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:50:37.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:50:37.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:50:37.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:50:37.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:50:37.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:50:37.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:50:37.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:50:37.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: [01/Oct/2023:19:50:36] ENGINE Bus STARTING 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: [01/Oct/2023:19:50:36] ENGINE Serving on http://172.21.15.87:8765 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:50:37.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:36 smithi087 ceph-mon[109386]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:50:37.264 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:37.071Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=5 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T19:50:37.568 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:37.263+0000 7f96998e3000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T19:50:37.874 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:37.703+0000 7f96998e3000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T19:50:37.874 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:37.569Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=5 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:50:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: [01/Oct/2023:19:50:36] ENGINE Serving on https://172.21.15.87:7150 2023-10-01T19:50:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: [01/Oct/2023:19:50:36] ENGINE Bus STARTED 2023-10-01T19:50:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: mgrmap e24: x(active, since 1.59489s) 2023-10-01T19:50:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T19:50:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T19:50:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:50:38.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:37 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:50:38.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: [01/Oct/2023:19:50:36] ENGINE Serving on https://172.21.15.87:7150 2023-10-01T19:50:38.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: [01/Oct/2023:19:50:36] ENGINE Bus STARTED 2023-10-01T19:50:38.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: mgrmap e24: x(active, since 1.59489s) 2023-10-01T19:50:38.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:38.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T19:50:38.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T19:50:38.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:38.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:50:38.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:50:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: [01/Oct/2023:19:50:36] ENGINE Serving on https://172.21.15.87:7150 2023-10-01T19:50:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: [01/Oct/2023:19:50:36] ENGINE Bus STARTED 2023-10-01T19:50:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: mgrmap e24: x(active, since 1.59489s) 2023-10-01T19:50:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T19:50:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T19:50:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:38.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:50:38.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:37 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:50:38.875 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:38 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:38.455+0000 7f96998e3000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T19:50:38.875 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:38 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:38.650+0000 7f96998e3000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T19:50:39.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:38 smithi087 ceph-mon[109386]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:39.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:38 smithi087 ceph-mon[109386]: mgrmap e25: x(active, since 2s) 2023-10-01T19:50:39.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:38 smithi087 ceph-mon[109386]: Reconfiguring daemon agent.smithi057 on smithi057 2023-10-01T19:50:39.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:38 smithi087 ceph-mon[109386]: Deploying cephadm binary to smithi057 2023-10-01T19:50:39.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:38 smithi057 ceph-mon[108286]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:39.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:38 smithi057 ceph-mon[108286]: mgrmap e25: x(active, since 2s) 2023-10-01T19:50:39.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:38 smithi057 ceph-mon[108286]: Reconfiguring daemon agent.smithi057 on smithi057 2023-10-01T19:50:39.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:38 smithi057 ceph-mon[108286]: Deploying cephadm binary to smithi057 2023-10-01T19:50:39.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:38 smithi057 ceph-mon[114331]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:39.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:38 smithi057 ceph-mon[114331]: mgrmap e25: x(active, since 2s) 2023-10-01T19:50:39.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:38 smithi057 ceph-mon[114331]: Reconfiguring daemon agent.smithi057 on smithi057 2023-10-01T19:50:39.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:38 smithi057 ceph-mon[114331]: Deploying cephadm binary to smithi057 2023-10-01T19:50:39.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:39 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:39.357+0000 7f96998e3000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T19:50:39.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:39 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:39.455+0000 7f96998e3000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T19:50:39.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:39 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:39.573+0000 7f96998e3000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T19:50:39.945 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:39 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:39.690+0000 7f96998e3000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T19:50:39.945 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:39 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:39.812+0000 7f96998e3000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T19:50:40.209 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:39 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:39.945+0000 7f96998e3000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T19:50:40.210 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:39 smithi057 systemd[1]: Stopping Ceph node-exporter.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:50:40.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:50:39 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:50:39] "GET /metrics HTTP/1.1" 200 34888 "" "Prometheus/2.33.4" 2023-10-01T19:50:40.582 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:40.327+0000 7f96998e3000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T19:50:40.582 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:40.439+0000 7f96998e3000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T19:50:40.583 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:40 smithi057 bash[155094]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter.a" found: no such container 2023-10-01T19:50:40.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:40.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:40.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:40.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:40 smithi087 ceph-mon[109386]: Reconfiguring node-exporter.a (dependencies changed)... 2023-10-01T19:50:40.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:40 smithi087 ceph-mon[109386]: Reconfiguring daemon node-exporter.a on smithi057 2023-10-01T19:50:40.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:40 smithi087 ceph-mon[109386]: mgrmap e26: x(active, since 4s) 2023-10-01T19:50:40.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:40.653+0000 7f96998e3000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T19:50:40.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:40.750+0000 7f96998e3000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T19:50:40.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:40.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:40.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:40.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-mon[108286]: Reconfiguring node-exporter.a (dependencies changed)... 2023-10-01T19:50:40.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-mon[108286]: Reconfiguring daemon node-exporter.a on smithi057 2023-10-01T19:50:40.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-mon[108286]: mgrmap e26: x(active, since 4s) 2023-10-01T19:50:40.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:40.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:40.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:40.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-mon[114331]: Reconfiguring node-exporter.a (dependencies changed)... 2023-10-01T19:50:40.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-mon[114331]: Reconfiguring daemon node-exporter.a on smithi057 2023-10-01T19:50:40.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:40 smithi057 ceph-mon[114331]: mgrmap e26: x(active, since 4s) 2023-10-01T19:50:41.421 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 bash[155146]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a 2023-10-01T19:50:41.421 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2023-10-01T19:50:41.422 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 bash[155246]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter.a" found: no such container 2023-10-01T19:50:41.422 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.a.service: Failed with result 'exit-code'. 2023-10-01T19:50:41.422 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 systemd[1]: Stopped Ceph node-exporter.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:50:41.422 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 systemd[1]: Starting Ceph node-exporter.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:50:41.885 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:41.600+0000 7f96998e3000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T19:50:41.885 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:41.825+0000 7f96998e3000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T19:50:41.885 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-mon[108286]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:41.886 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-mon[114331]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:42.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:41 smithi087 ceph-mon[109386]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:42.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:42.027+0000 7f96998e3000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:50:42.172 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.885Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2023-10-01T19:50:42.172 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.885Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2023-10-01T19:50:42.172 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.885Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2023-10-01T19:50:42.172 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.885Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2023-10-01T19:50:42.172 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2023-10-01T19:50:42.172 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=arp 2023-10-01T19:50:42.172 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=bcache 2023-10-01T19:50:42.173 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=bonding 2023-10-01T19:50:42.173 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=btrfs 2023-10-01T19:50:42.173 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=conntrack 2023-10-01T19:50:42.173 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=cpu 2023-10-01T19:50:42.173 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=cpufreq 2023-10-01T19:50:42.173 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=diskstats 2023-10-01T19:50:42.173 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=dmi 2023-10-01T19:50:42.173 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=edac 2023-10-01T19:50:42.173 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=entropy 2023-10-01T19:50:42.173 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=fibrechannel 2023-10-01T19:50:42.173 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=filefd 2023-10-01T19:50:42.173 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=filesystem 2023-10-01T19:50:42.174 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=hwmon 2023-10-01T19:50:42.174 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=infiniband 2023-10-01T19:50:42.174 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=ipvs 2023-10-01T19:50:42.174 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=loadavg 2023-10-01T19:50:42.174 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=mdadm 2023-10-01T19:50:42.174 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=meminfo 2023-10-01T19:50:42.174 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=netclass 2023-10-01T19:50:42.174 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=netdev 2023-10-01T19:50:42.174 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=netstat 2023-10-01T19:50:42.174 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=nfs 2023-10-01T19:50:42.174 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=nfsd 2023-10-01T19:50:42.174 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=nvme 2023-10-01T19:50:42.175 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=os 2023-10-01T19:50:42.175 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2023-10-01T19:50:42.175 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=pressure 2023-10-01T19:50:42.175 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=rapl 2023-10-01T19:50:42.175 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=schedstat 2023-10-01T19:50:42.175 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=sockstat 2023-10-01T19:50:42.175 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=softnet 2023-10-01T19:50:42.175 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=stat 2023-10-01T19:50:42.175 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=tapestats 2023-10-01T19:50:42.175 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=textfile 2023-10-01T19:50:42.175 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=thermal_zone 2023-10-01T19:50:42.175 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=time 2023-10-01T19:50:42.176 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=udp_queues 2023-10-01T19:50:42.176 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=uname 2023-10-01T19:50:42.176 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=vmstat 2023-10-01T19:50:42.176 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=xfs 2023-10-01T19:50:42.176 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:115 level=info collector=zfs 2023-10-01T19:50:42.176 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.887Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2023-10-01T19:50:42.176 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[155373]: ts=2023-10-01T19:50:41.888Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2023-10-01T19:50:42.176 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 bash[155355]: 41af659015b7ce38f3950f2d378c0e54b0c74eb12aef4416851e72271c33e65f 2023-10-01T19:50:42.176 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 19:50:41 smithi057 systemd[1]: Started Ceph node-exporter.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:50:42.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:42.521+0000 7f96998e3000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T19:50:42.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:42.707+0000 7f96998e3000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:50:43.284 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:43.189+0000 7f96998e3000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T19:50:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-mon[108286]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-mon[108286]: Reconfiguring alertmanager.a (dependencies changed)... 2023-10-01T19:50:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-mon[108286]: Reconfiguring daemon alertmanager.a on smithi057 2023-10-01T19:50:43.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-mon[114331]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:43.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:43.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-mon[114331]: Reconfiguring alertmanager.a (dependencies changed)... 2023-10-01T19:50:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:42 smithi057 ceph-mon[114331]: Reconfiguring daemon alertmanager.a on smithi057 2023-10-01T19:50:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:42 smithi087 ceph-mon[109386]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:50:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:42 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:42 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:42 smithi087 ceph-mon[109386]: Reconfiguring alertmanager.a (dependencies changed)... 2023-10-01T19:50:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:42 smithi087 ceph-mon[109386]: Reconfiguring daemon alertmanager.a on smithi057 2023-10-01T19:50:43.580 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=error ts=2023-10-01T19:50:43.519Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=8 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.57:8443//api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T19:50:43.580 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:43.521Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.87:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.87 because it doesn't contain any IP SANs" 2023-10-01T19:50:43.580 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=warn ts=2023-10-01T19:50:43.522Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.57:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.57 because it doesn't contain any IP SANs" 2023-10-01T19:50:43.581 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:43.333+0000 7f96998e3000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T19:50:43.581 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:50:43] ENGINE Bus STARTING 2023-10-01T19:50:43.581 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: CherryPy Checker: 2023-10-01T19:50:43.581 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: The Application mounted at '' has an empty config. 2023-10-01T19:50:43.581 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: 2023-10-01T19:50:43.581 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:50:43] ENGINE Serving on http://:::9283 2023-10-01T19:50:43.581 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: [01/Oct/2023:19:50:43] ENGINE Bus STARTED 2023-10-01T19:50:44.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-mon[108286]: Standby manager daemon y started 2023-10-01T19:50:44.361 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.57:0/773030616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-01T19:50:44.361 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.57:0/773030616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:50:44.361 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.57:0/773030616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-01T19:50:44.361 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.57:0/773030616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:50:44.361 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-mon[114331]: Standby manager daemon y started 2023-10-01T19:50:44.361 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.57:0/773030616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-01T19:50:44.361 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.57:0/773030616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:50:44.361 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.57:0/773030616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-01T19:50:44.362 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:43 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.57:0/773030616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:50:44.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:43 smithi087 ceph-mon[109386]: Standby manager daemon y started 2023-10-01T19:50:44.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:43 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.57:0/773030616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-01T19:50:44.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:43 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.57:0/773030616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:50:44.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:43 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.57:0/773030616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-01T19:50:44.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:43 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.57:0/773030616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:50:45.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:44 smithi087 ceph-mon[109386]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T19:50:45.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:44 smithi087 ceph-mon[109386]: mgrmap e27: x(active, since 8s), standbys: y 2023-10-01T19:50:45.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:44 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:50:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:44 smithi057 ceph-mon[114331]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T19:50:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:44 smithi057 ceph-mon[114331]: mgrmap e27: x(active, since 8s), standbys: y 2023-10-01T19:50:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:44 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:50:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:44 smithi057 ceph-mon[108286]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T19:50:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:44 smithi057 ceph-mon[108286]: mgrmap e27: x(active, since 8s), standbys: y 2023-10-01T19:50:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:44 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:50:46.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:50:46] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:50:47.322 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:46 smithi057 ceph-mon[108286]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2023-10-01T19:50:47.322 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:46 smithi057 ceph-mon[114331]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2023-10-01T19:50:47.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:46 smithi087 ceph-mon[109386]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2023-10-01T19:50:48.320 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:47 smithi057 ceph-mon[108286]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:48.320 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:47 smithi057 ceph-mon[114331]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:47 smithi087 ceph-mon[109386]: from='client.24749 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:50:48.641 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:48 smithi057 systemd[1]: Stopping Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:50:48.642 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:48 smithi057 bash[156201]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager.a" found: no such container 2023-10-01T19:50:48.642 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:48 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[138918]: level=info ts=2023-10-01T19:50:48.528Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2023-10-01T19:50:48.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:48 smithi057 bash[156213]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a 2023-10-01T19:50:48.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:48 smithi057 bash[156249]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager.a" found: no such container 2023-10-01T19:50:48.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:48 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@alertmanager.a.service: Succeeded. 2023-10-01T19:50:48.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:48 smithi057 systemd[1]: Stopped Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:50:48.923 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:48 smithi057 systemd[1]: Starting Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:50:49.270 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:48 smithi057 ceph-mon[108286]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T19:50:49.270 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:48 smithi057 ceph-mon[114331]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T19:50:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:48 smithi087 ceph-mon[109386]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T19:50:49.671 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[156331]: level=info ts=2023-10-01T19:50:49.377Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2023-10-01T19:50:49.671 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[156331]: level=info ts=2023-10-01T19:50:49.378Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2023-10-01T19:50:49.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[156331]: level=info ts=2023-10-01T19:50:49.379Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.57 port=9094 2023-10-01T19:50:49.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[156331]: level=info ts=2023-10-01T19:50:49.381Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2023-10-01T19:50:49.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[156331]: level=info ts=2023-10-01T19:50:49.409Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2023-10-01T19:50:49.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[156331]: level=info ts=2023-10-01T19:50:49.409Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2023-10-01T19:50:49.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[156331]: level=info ts=2023-10-01T19:50:49.412Z caller=main.go:518 msg=Listening address=:9093 2023-10-01T19:50:49.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[156331]: level=info ts=2023-10-01T19:50:49.412Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2023-10-01T19:50:49.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:49 smithi057 bash[156317]: 0b16afb678d95121bc251f9294e84d9deaa644884e5f3eb65959680c8c0ae7c2 2023-10-01T19:50:49.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:49 smithi057 systemd[1]: Started Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:50:50.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:50:49 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:50:49] "GET /metrics HTTP/1.1" 200 34888 "" "Prometheus/2.33.4" 2023-10-01T19:50:50.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:50.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:50.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[108286]: Reconfiguring iscsi.foo.smithi057.zfdcts (dependencies changed)... 2023-10-01T19:50:50.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T19:50:50.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T19:50:50.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:50:50.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[108286]: Reconfiguring daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T19:50:50.530 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:50.530 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:50.530 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[114331]: Reconfiguring iscsi.foo.smithi057.zfdcts (dependencies changed)... 2023-10-01T19:50:50.530 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T19:50:50.530 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T19:50:50.530 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:50:50.531 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:50 smithi057 ceph-mon[114331]: Reconfiguring daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T19:50:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:50 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:50 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:50 smithi087 ceph-mon[109386]: Reconfiguring iscsi.foo.smithi057.zfdcts (dependencies changed)... 2023-10-01T19:50:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:50 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T19:50:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:50 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T19:50:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:50 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:50:50.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:50 smithi087 ceph-mon[109386]: Reconfiguring daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T19:50:51.421 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:51 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[156331]: level=info ts=2023-10-01T19:50:51.381Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000282892s 2023-10-01T19:50:51.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:51 smithi057 ceph-mon[108286]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T19:50:51.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:51 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:50:51.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:51 smithi057 ceph-mon[114331]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T19:50:51.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:51 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:50:52.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:51 smithi087 ceph-mon[109386]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T19:50:52.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:51 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:50:52.632 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:50:52 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:50:52.314Z caller=manager.go:609 level=warn component="rule manager" group=pools msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on(pool_id) group_right()\n ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: |\n Pool '{{ $labels.name }}' will be full in less than 5 days assuming the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed it's capacity\n" err="found duplicate series for the match group {pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"172.21.15.87:9283\", job=\"ceph\", pool_id=\"1\"}, {instance=\"172.21.15.57:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:50:53.199 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:53 smithi087 ceph-mon[109386]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T19:50:53.199 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:53 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:53.199 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:53 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:53.199 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:53 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:50:53.199 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:53 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:50:53.200 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:53 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2447207527' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T19:50:53.200 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:53 smithi087 ceph-mon[109386]: Reconfiguring daemon agent.smithi087 on smithi087 2023-10-01T19:50:53.200 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:53 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/136105658' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2843362731"}]: dispatch 2023-10-01T19:50:53.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[108286]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T19:50:53.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:53.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:53.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:50:53.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:50:53.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2447207527' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T19:50:53.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[108286]: Reconfiguring daemon agent.smithi087 on smithi087 2023-10-01T19:50:53.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/136105658' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2843362731"}]: dispatch 2023-10-01T19:50:53.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[114331]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T19:50:53.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:53.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:53.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:50:53.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:50:53.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2447207527' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T19:50:53.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[114331]: Reconfiguring daemon agent.smithi087 on smithi087 2023-10-01T19:50:53.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:53 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/136105658' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2843362731"}]: dispatch 2023-10-01T19:50:54.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:54 smithi057 ceph-mon[114331]: Deploying cephadm binary to smithi087 2023-10-01T19:50:54.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:54 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/136105658' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2843362731"}]': finished 2023-10-01T19:50:54.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:54 smithi057 ceph-mon[114331]: osdmap e92: 8 total, 8 up, 8 in 2023-10-01T19:50:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:54 smithi057 ceph-mon[108286]: Deploying cephadm binary to smithi087 2023-10-01T19:50:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:54 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/136105658' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2843362731"}]': finished 2023-10-01T19:50:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:54 smithi057 ceph-mon[108286]: osdmap e92: 8 total, 8 up, 8 in 2023-10-01T19:50:54.524 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:54 smithi087 ceph-mon[109386]: Deploying cephadm binary to smithi087 2023-10-01T19:50:54.524 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:54 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/136105658' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2843362731"}]': finished 2023-10-01T19:50:54.524 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:54 smithi087 ceph-mon[109386]: osdmap e92: 8 total, 8 up, 8 in 2023-10-01T19:50:55.224 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:55 smithi087 ceph-mon[109386]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2023-10-01T19:50:55.224 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:55 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2886871262' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2722579079"}]: dispatch 2023-10-01T19:50:55.224 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:55 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:55.224 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:55 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:55.224 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:55 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:55.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:55 smithi057 ceph-mon[114331]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2023-10-01T19:50:55.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:55 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2886871262' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2722579079"}]: dispatch 2023-10-01T19:50:55.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:55 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:55.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:55 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:55.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:55 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:55.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:55 smithi057 ceph-mon[108286]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2023-10-01T19:50:55.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:55 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2886871262' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2722579079"}]: dispatch 2023-10-01T19:50:55.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:55 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:55.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:55 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:55.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:55 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:50:56.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[108533]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:50:56] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2023-10-01T19:50:56.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-mon[108286]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T19:50:56.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-mon[108286]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T19:50:56.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2886871262' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2722579079"}]': finished 2023-10-01T19:50:56.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-mon[108286]: osdmap e93: 8 total, 8 up, 8 in 2023-10-01T19:50:56.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1952043369' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2722579079"}]: dispatch 2023-10-01T19:50:56.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2722579079"}]: dispatch 2023-10-01T19:50:56.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-mon[114331]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T19:50:56.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-mon[114331]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T19:50:56.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2886871262' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2722579079"}]': finished 2023-10-01T19:50:56.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-mon[114331]: osdmap e93: 8 total, 8 up, 8 in 2023-10-01T19:50:56.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1952043369' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2722579079"}]: dispatch 2023-10-01T19:50:56.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:56 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2722579079"}]: dispatch 2023-10-01T19:50:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:56 smithi087 ceph-mon[109386]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T19:50:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:56 smithi087 ceph-mon[109386]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T19:50:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:56 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2886871262' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2722579079"}]': finished 2023-10-01T19:50:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:56 smithi087 ceph-mon[109386]: osdmap e93: 8 total, 8 up, 8 in 2023-10-01T19:50:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:56 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1952043369' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2722579079"}]: dispatch 2023-10-01T19:50:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:56 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2722579079"}]: dispatch 2023-10-01T19:50:57.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:57 smithi087 ceph-mon[109386]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2023-10-01T19:50:57.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:57 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2722579079"}]': finished 2023-10-01T19:50:57.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:57 smithi087 ceph-mon[109386]: osdmap e94: 8 total, 8 up, 8 in 2023-10-01T19:50:57.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:57 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3403937727' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/560615295"}]: dispatch 2023-10-01T19:50:57.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:57 smithi057 ceph-mon[114331]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2023-10-01T19:50:57.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:57 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2722579079"}]': finished 2023-10-01T19:50:57.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:57 smithi057 ceph-mon[114331]: osdmap e94: 8 total, 8 up, 8 in 2023-10-01T19:50:57.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:57 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3403937727' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/560615295"}]: dispatch 2023-10-01T19:50:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:57 smithi057 ceph-mon[108286]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 67 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2023-10-01T19:50:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:57 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2722579079"}]': finished 2023-10-01T19:50:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:57 smithi057 ceph-mon[108286]: osdmap e94: 8 total, 8 up, 8 in 2023-10-01T19:50:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:57 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3403937727' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/560615295"}]: dispatch 2023-10-01T19:50:58.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:58 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3403937727' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/560615295"}]': finished 2023-10-01T19:50:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:58 smithi087 ceph-mon[109386]: osdmap e95: 8 total, 8 up, 8 in 2023-10-01T19:50:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:58 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/158368003' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/286188666"}]: dispatch 2023-10-01T19:50:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:58 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/286188666"}]: dispatch 2023-10-01T19:50:58.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:58 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3403937727' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/560615295"}]': finished 2023-10-01T19:50:58.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:58 smithi057 ceph-mon[108286]: osdmap e95: 8 total, 8 up, 8 in 2023-10-01T19:50:58.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:58 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/158368003' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/286188666"}]: dispatch 2023-10-01T19:50:58.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:58 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/286188666"}]: dispatch 2023-10-01T19:50:58.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:58 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3403937727' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/560615295"}]': finished 2023-10-01T19:50:58.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:58 smithi057 ceph-mon[114331]: osdmap e95: 8 total, 8 up, 8 in 2023-10-01T19:50:58.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:58 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/158368003' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/286188666"}]: dispatch 2023-10-01T19:50:58.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:58 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/286188666"}]: dispatch 2023-10-01T19:50:59.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:59 smithi087 ceph-mon[109386]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:59.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:59 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/286188666"}]': finished 2023-10-01T19:50:59.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:59 smithi087 ceph-mon[109386]: osdmap e96: 8 total, 8 up, 8 in 2023-10-01T19:50:59.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:59 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/176992053' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/418424420"}]: dispatch 2023-10-01T19:50:59.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:50:59 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/418424420"}]: dispatch 2023-10-01T19:50:59.673 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 19:50:59 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[156331]: level=info ts=2023-10-01T19:50:59.384Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.003254778s 2023-10-01T19:50:59.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:59 smithi057 ceph-mon[108286]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:59.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:59 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/286188666"}]': finished 2023-10-01T19:50:59.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:59 smithi057 ceph-mon[108286]: osdmap e96: 8 total, 8 up, 8 in 2023-10-01T19:50:59.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:59 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/176992053' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/418424420"}]: dispatch 2023-10-01T19:50:59.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:50:59 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/418424420"}]: dispatch 2023-10-01T19:50:59.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:59 smithi057 ceph-mon[114331]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:50:59.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:59 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/286188666"}]': finished 2023-10-01T19:50:59.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:59 smithi057 ceph-mon[114331]: osdmap e96: 8 total, 8 up, 8 in 2023-10-01T19:50:59.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:59 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/176992053' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/418424420"}]: dispatch 2023-10-01T19:50:59.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:50:59 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/418424420"}]: dispatch 2023-10-01T19:51:00.221 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:50:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:50:59] "GET /metrics HTTP/1.1" 200 37882 "" "Prometheus/2.33.4" 2023-10-01T19:51:00.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:00 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/418424420"}]': finished 2023-10-01T19:51:00.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:00 smithi057 ceph-mon[108286]: osdmap e97: 8 total, 8 up, 8 in 2023-10-01T19:51:00.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:00 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2271383551' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3003758368"}]: dispatch 2023-10-01T19:51:00.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:00 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3003758368"}]: dispatch 2023-10-01T19:51:00.609 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:00 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/418424420"}]': finished 2023-10-01T19:51:00.609 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:00 smithi057 ceph-mon[114331]: osdmap e97: 8 total, 8 up, 8 in 2023-10-01T19:51:00.609 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:00 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2271383551' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3003758368"}]: dispatch 2023-10-01T19:51:00.609 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:00 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3003758368"}]: dispatch 2023-10-01T19:51:00.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:00 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/418424420"}]': finished 2023-10-01T19:51:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:00 smithi087 ceph-mon[109386]: osdmap e97: 8 total, 8 up, 8 in 2023-10-01T19:51:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:00 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2271383551' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3003758368"}]: dispatch 2023-10-01T19:51:00.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:00 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3003758368"}]: dispatch 2023-10-01T19:51:01.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:01 smithi057 ceph-mon[108286]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:01.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:01 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3003758368"}]': finished 2023-10-01T19:51:01.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:01 smithi057 ceph-mon[108286]: osdmap e98: 8 total, 8 up, 8 in 2023-10-01T19:51:01.419 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:01 smithi057 ceph-mon[114331]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:01.419 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:01 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3003758368"}]': finished 2023-10-01T19:51:01.419 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:01 smithi057 ceph-mon[114331]: osdmap e98: 8 total, 8 up, 8 in 2023-10-01T19:51:01.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:01 smithi087 ceph-mon[109386]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:01.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:01 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3003758368"}]': finished 2023-10-01T19:51:01.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:01 smithi087 ceph-mon[109386]: osdmap e98: 8 total, 8 up, 8 in 2023-10-01T19:51:02.217 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 systemd[1]: Stopping Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:51:02.474 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 bash[142460]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus.a" found: no such container 2023-10-01T19:51:02.475 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:51:02.296Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2023-10-01T19:51:02.475 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:51:02.297Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2023-10-01T19:51:02.475 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:51:02.297Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2023-10-01T19:51:02.475 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:51:02.297Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2023-10-01T19:51:02.475 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:51:02.297Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2023-10-01T19:51:02.475 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:51:02.297Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2023-10-01T19:51:02.475 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:51:02.297Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2023-10-01T19:51:02.475 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:51:02.297Z caller=main.go:828 level=info msg="Scrape manager stopped" 2023-10-01T19:51:02.475 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:51:02.297Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2023-10-01T19:51:02.475 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:51:02.298Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2023-10-01T19:51:02.476 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:51:02.298Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2023-10-01T19:51:02.476 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[131146]: ts=2023-10-01T19:51:02.298Z caller=main.go:1066 level=info msg="See you next time!" 2023-10-01T19:51:02.476 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 bash[142472]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a 2023-10-01T19:51:02.476 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 bash[142509]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus.a" found: no such container 2023-10-01T19:51:02.809 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@prometheus.a.service: Succeeded. 2023-10-01T19:51:02.809 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 systemd[1]: Stopped Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:51:02.809 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 systemd[1]: Starting Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:51:03.241 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 bash[142576]: f7dd35d9dddbbdb2ed7bb2c0ac76b5ee3def365e8e8735487d9dd54ce8c94aad 2023-10-01T19:51:03.242 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 systemd[1]: Started Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:51:03.242 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.926Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2023-10-01T19:51:03.242 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.926Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2023-10-01T19:51:03.242 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.926Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2023-10-01T19:51:03.242 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.926Z caller=main.go:518 level=info host_details="(Linux 4.18.0-514.el8.x86_64 #1 SMP Tue Sep 19 13:19:58 UTC 2023 x86_64 smithi087 (none))" 2023-10-01T19:51:03.242 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.926Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2023-10-01T19:51:03.242 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.926Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2023-10-01T19:51:03.243 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.928Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2023-10-01T19:51:03.243 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.929Z caller=main.go:923 level=info msg="Starting TSDB ..." 2023-10-01T19:51:03.243 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.930Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2023-10-01T19:51:03.243 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.933Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2023-10-01T19:51:03.243 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.933Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.335µs 2023-10-01T19:51:03.243 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.933Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2023-10-01T19:51:03.243 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:02.934Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=2 2023-10-01T19:51:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:03 smithi087 ceph-mon[109386]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:51:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:03 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:03 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:03 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:03.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:03.366Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=2 2023-10-01T19:51:03.634 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:03.366Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=2 2023-10-01T19:51:03.634 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:03.366Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=33.858µs wal_replay_duration=433.201226ms total_replay_duration=433.249654ms 2023-10-01T19:51:03.634 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:03.371Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2023-10-01T19:51:03.634 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:03.371Z caller=main.go:947 level=info msg="TSDB started" 2023-10-01T19:51:03.634 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:03.371Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2023-10-01T19:51:03.634 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:03.391Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.561708ms db_storage=809ns remote_storage=1.732µs web_handler=403ns query_engine=825ns scrape=612.278µs scrape_sd=118.491µs notify=21.242µs notify_sd=17.551µs rules=19.307408ms 2023-10-01T19:51:03.635 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:03.391Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2023-10-01T19:51:03.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:03 smithi057 ceph-mon[114331]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:51:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:03 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:03 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:03 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:03 smithi057 ceph-mon[108286]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:51:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:03 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:03 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:03 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:04.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-mon[109386]: Reconfiguring node-exporter.b (dependencies changed)... 2023-10-01T19:51:04.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-mon[109386]: Reconfiguring daemon node-exporter.b on smithi087 2023-10-01T19:51:04.383 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 systemd[1]: Stopping Ceph node-exporter.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:51:04.383 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 bash[142837]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter.b" found: no such container 2023-10-01T19:51:04.383 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 bash[142856]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b 2023-10-01T19:51:04.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:04 smithi057 ceph-mon[108286]: Reconfiguring node-exporter.b (dependencies changed)... 2023-10-01T19:51:04.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:04 smithi057 ceph-mon[108286]: Reconfiguring daemon node-exporter.b on smithi087 2023-10-01T19:51:04.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:04 smithi057 ceph-mon[114331]: Reconfiguring node-exporter.b (dependencies changed)... 2023-10-01T19:51:04.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:04 smithi057 ceph-mon[114331]: Reconfiguring daemon node-exporter.b on smithi087 2023-10-01T19:51:04.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2023-10-01T19:51:04.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 bash[142893]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter.b" found: no such container 2023-10-01T19:51:04.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.b.service: Failed with result 'exit-code'. 2023-10-01T19:51:04.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 systemd[1]: Stopped Ceph node-exporter.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:51:04.663 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 systemd[1]: Starting Ceph node-exporter.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:51:05.133 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.763Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2023-10-01T19:51:05.133 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.763Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2023-10-01T19:51:05.133 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.763Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2023-10-01T19:51:05.133 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.763Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2023-10-01T19:51:05.134 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2023-10-01T19:51:05.134 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=arp 2023-10-01T19:51:05.134 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=bcache 2023-10-01T19:51:05.134 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=bonding 2023-10-01T19:51:05.134 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=btrfs 2023-10-01T19:51:05.134 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=conntrack 2023-10-01T19:51:05.134 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=cpu 2023-10-01T19:51:05.134 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=cpufreq 2023-10-01T19:51:05.134 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=diskstats 2023-10-01T19:51:05.134 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=dmi 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=edac 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=entropy 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=fibrechannel 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=filefd 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=filesystem 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=hwmon 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=infiniband 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=ipvs 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=loadavg 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=mdadm 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=meminfo 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=netclass 2023-10-01T19:51:05.135 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=netdev 2023-10-01T19:51:05.136 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=netstat 2023-10-01T19:51:05.136 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=nfs 2023-10-01T19:51:05.136 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=nfsd 2023-10-01T19:51:05.136 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=nvme 2023-10-01T19:51:05.136 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=os 2023-10-01T19:51:05.136 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2023-10-01T19:51:05.136 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=pressure 2023-10-01T19:51:05.136 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=rapl 2023-10-01T19:51:05.136 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=schedstat 2023-10-01T19:51:05.136 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=sockstat 2023-10-01T19:51:05.136 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=softnet 2023-10-01T19:51:05.137 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=stat 2023-10-01T19:51:05.137 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=tapestats 2023-10-01T19:51:05.137 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=textfile 2023-10-01T19:51:05.137 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=thermal_zone 2023-10-01T19:51:05.137 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=time 2023-10-01T19:51:05.137 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=udp_queues 2023-10-01T19:51:05.137 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=uname 2023-10-01T19:51:05.137 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=vmstat 2023-10-01T19:51:05.138 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=xfs 2023-10-01T19:51:05.138 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.764Z caller=node_exporter.go:115 level=info collector=zfs 2023-10-01T19:51:05.138 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.765Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2023-10-01T19:51:05.138 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[142975]: ts=2023-10-01T19:51:04.765Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2023-10-01T19:51:05.138 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 bash[142960]: dd38d7fd60ea902f980b6575eb2757ad804542ae6f651f7b2896fc28d21e55ee 2023-10-01T19:51:05.138 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 19:51:04 smithi087 systemd[1]: Started Ceph node-exporter.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:51:06.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:05 smithi087 ceph-mon[109386]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:06.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:05 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:06.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:05 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:06.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:05 smithi087 ceph-mon[109386]: Reconfiguring grafana.a (dependencies changed)... 2023-10-01T19:51:06.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:05 smithi087 ceph-mon[109386]: Regenerating cephadm self-signed grafana TLS certificates 2023-10-01T19:51:06.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:05 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:06.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:05 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:06.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:05 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-10-01T19:51:06.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:05 smithi087 ceph-mon[109386]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-10-01T19:51:06.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:05 smithi087 ceph-mon[109386]: Reconfiguring daemon grafana.a on smithi087 2023-10-01T19:51:06.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[108286]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:06.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:06.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:06.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[108286]: Reconfiguring grafana.a (dependencies changed)... 2023-10-01T19:51:06.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[108286]: Regenerating cephadm self-signed grafana TLS certificates 2023-10-01T19:51:06.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:06.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:06.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-10-01T19:51:06.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[108286]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-10-01T19:51:06.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[108286]: Reconfiguring daemon grafana.a on smithi087 2023-10-01T19:51:06.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[114331]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:06.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:06.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:06.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[114331]: Reconfiguring grafana.a (dependencies changed)... 2023-10-01T19:51:06.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[114331]: Regenerating cephadm self-signed grafana TLS certificates 2023-10-01T19:51:06.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:06.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:06.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-10-01T19:51:06.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[114331]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-10-01T19:51:06.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:05 smithi057 ceph-mon[114331]: Reconfiguring daemon grafana.a on smithi087 2023-10-01T19:51:07.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:06 smithi087 ceph-mon[109386]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 669 B/s rd, 0 op/s 2023-10-01T19:51:07.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:06 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:51:07.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:06 smithi057 ceph-mon[108286]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 669 B/s rd, 0 op/s 2023-10-01T19:51:07.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:06 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:51:07.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:06 smithi057 ceph-mon[114331]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 669 B/s rd, 0 op/s 2023-10-01T19:51:07.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:06 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:51:09.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:08 smithi057 ceph-mon[108286]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:09.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:08 smithi057 ceph-mon[114331]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:09.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:08 smithi087 ceph-mon[109386]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:11.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:10 smithi087 ceph-mon[109386]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-10-01T19:51:11.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:51:10] "GET /metrics HTTP/1.1" 200 37879 "" "Prometheus/2.33.4" 2023-10-01T19:51:11.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:10 smithi057 ceph-mon[108286]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-10-01T19:51:11.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:10 smithi057 ceph-mon[114331]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-10-01T19:51:13.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:12 smithi057 ceph-mon[114331]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 879 B/s rd, 0 op/s 2023-10-01T19:51:13.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:12 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:13.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:12 smithi057 ceph-mon[108286]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 879 B/s rd, 0 op/s 2023-10-01T19:51:13.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:12 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:13.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:12 smithi087 ceph-mon[109386]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 879 B/s rd, 0 op/s 2023-10-01T19:51:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:12 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:15.975 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:15 smithi087 ceph-mon[109386]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:15.976 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:15 smithi087 systemd[1]: Stopping Ceph grafana.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:51:15.976 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:15 smithi087 bash[143345]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana.a" found: no such container 2023-10-01T19:51:16.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:15 smithi057 ceph-mon[108286]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:16.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:15 smithi057 ceph-mon[114331]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:16.237 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:15 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[129938]: t=2023-10-01T19:51:15+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2023-10-01T19:51:16.237 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 bash[143363]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a 2023-10-01T19:51:16.238 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 bash[143446]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana.a" found: no such container 2023-10-01T19:51:16.498 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@grafana.a.service: Succeeded. 2023-10-01T19:51:16.499 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 systemd[1]: Stopped Ceph grafana.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:51:16.499 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 systemd[1]: Starting Ceph grafana.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:51:16.947 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="The state of unified alerting is still not defined. The decision will be made during as we run the database migrations" logger=settings 2023-10-01T19:51:16.947 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=warn msg="falling back to legacy setting of 'min_interval_seconds'; please use the configuration option in the `unified_alerting` section if Grafana 8 alerts are enabled." logger=settings 2023-10-01T19:51:16.947 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini 2023-10-01T19:51:16.949 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini 2023-10-01T19:51:16.949 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_DATA=/var/lib/grafana" 2023-10-01T19:51:16.949 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_LOGS=/var/log/grafana" 2023-10-01T19:51:16.949 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2023-10-01T19:51:16.949 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2023-10-01T19:51:16.949 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana 2023-10-01T19:51:16.949 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana 2023-10-01T19:51:16.949 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana 2023-10-01T19:51:16.950 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins 2023-10-01T19:51:16.950 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning 2023-10-01T19:51:16.950 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="App mode production" logger=settings 2023-10-01T19:51:16.950 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2023-10-01T19:51:16.950 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=warn msg="SQLite database file has broader permissions than it should" logger=sqlstore path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2023-10-01T19:51:16.950 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Starting DB migrations" logger=migrator 2023-10-01T19:51:16.950 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 bash[143542]: ca8ad8b1ccf83c68e6688b720a328b08dae211cb8246319a3b9b993624be4294 2023-10-01T19:51:16.950 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="migrations completed" logger=migrator performed=0 skipped=377 duration=532.716µs 2023-10-01T19:51:16.950 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Created default organization" logger=sqlstore 2023-10-01T19:51:16.951 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:16] ENGINE Bus STOPPING 2023-10-01T19:51:16.951 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:16] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-10-01T19:51:16.951 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:16] ENGINE Bus STOPPED 2023-10-01T19:51:16.951 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:16] ENGINE Bus STARTING 2023-10-01T19:51:16.951 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:16] ENGINE Serving on http://:::9283 2023-10-01T19:51:16.951 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:16] ENGINE Bus STARTED 2023-10-01T19:51:16.951 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:16] ENGINE Bus STOPPING 2023-10-01T19:51:16.951 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:16] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-10-01T19:51:16.951 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:16] ENGINE Bus STOPPED 2023-10-01T19:51:16.951 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:16] ENGINE Bus STARTING 2023-10-01T19:51:16.952 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Initialising plugins" logger=plugin.manager 2023-10-01T19:51:16.952 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 systemd[1]: Started Ceph grafana.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:51:16.952 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=input 2023-10-01T19:51:16.952 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=grafana-piechart-panel 2023-10-01T19:51:16.952 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=vonage-status-panel 2023-10-01T19:51:16.952 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="Live Push Gateway initialization" logger=live.push_http 2023-10-01T19:51:16.952 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="deleted datasource based on configuration" logger=provisioning.datasources name=Dashboard1 2023-10-01T19:51:16.952 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 uid=P43CA22E17D0F9596 2023-10-01T19:51:16.953 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Loki uid=P8E80F9AEF21F6940 2023-10-01T19:51:16.953 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2023-10-01T19:51:16.953 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="warming cache for startup" logger=ngalert 2023-10-01T19:51:16.953 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 19:51:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T19:51:16+0000 lvl=info msg="starting MultiOrg Alertmanager" logger=ngalert.multiorg.alertmanager 2023-10-01T19:51:17.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:17] ENGINE Serving on http://:::9283 2023-10-01T19:51:17.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:17] ENGINE Bus STARTED 2023-10-01T19:51:17.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:17] ENGINE Bus STOPPING 2023-10-01T19:51:17.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:17] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-10-01T19:51:17.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:17] ENGINE Bus STOPPED 2023-10-01T19:51:17.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:17] ENGINE Bus STARTING 2023-10-01T19:51:17.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:17] ENGINE Serving on http://:::9283 2023-10-01T19:51:17.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:51:17] ENGINE Bus STARTED 2023-10-01T19:51:17.837 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:17.838 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.838 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.838 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T19:51:17.838 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T19:51:17.838 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2023-10-01T19:51:17.838 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2023-10-01T19:51:17.838 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.838 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:51:17.838 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:51:17.838 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.838 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: Adding iSCSI gateway http://:@172.21.15.57:5000 to Dashboard 2023-10-01T19:51:17.839 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:51:17.839 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:51:17.839 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:51:17.839 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:51:17.839 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.839 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:51:17.839 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:51:17.839 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi087.front.sepia.ceph.com:9095"}]: dispatch 2023-10-01T19:51:17.839 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi087.front.sepia.ceph.com:9095"}]: dispatch 2023-10-01T19:51:17.839 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.839 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T19:51:17.839 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi087.front.sepia.ceph.com:3000"}]: dispatch 2023-10-01T19:51:17.840 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.840 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:51:17.840 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:51:17.840 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:51:17.840 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:51:17.840 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:51:17.840 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.840 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:17.840 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T19:51:17.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T19:51:17.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2023-10-01T19:51:17.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2023-10-01T19:51:17.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:51:17.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:51:17.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: Adding iSCSI gateway http://:@172.21.15.57:5000 to Dashboard 2023-10-01T19:51:17.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:51:17.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:51:17.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:51:17.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:51:17.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:51:17.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:51:17.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi087.front.sepia.ceph.com:9095"}]: dispatch 2023-10-01T19:51:17.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi087.front.sepia.ceph.com:9095"}]: dispatch 2023-10-01T19:51:17.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T19:51:17.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi087.front.sepia.ceph.com:3000"}]: dispatch 2023-10-01T19:51:17.842 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:51:17.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:51:17.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:51:17.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:51:17.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:51:17.843 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:17 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:17.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T19:51:17.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T19:51:17.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2023-10-01T19:51:17.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2023-10-01T19:51:17.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:51:17.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:51:17.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: Adding iSCSI gateway http://:@172.21.15.57:5000 to Dashboard 2023-10-01T19:51:17.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:51:17.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:51:17.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:51:17.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:51:17.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:51:17.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:51:17.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi087.front.sepia.ceph.com:9095"}]: dispatch 2023-10-01T19:51:17.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi087.front.sepia.ceph.com:9095"}]: dispatch 2023-10-01T19:51:17.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T19:51:17.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi087.front.sepia.ceph.com:3000"}]: dispatch 2023-10-01T19:51:17.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:17.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:51:17.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:51:17.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:51:17.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:51:17.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:51:17.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:17 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:18.822 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T19:51:18.823 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi087.front.sepia.ceph.com:3000"}]: dispatch 2023-10-01T19:51:18.823 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: Deploying daemon agent.smithi057 on smithi057 2023-10-01T19:51:18.823 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: Deploying daemon agent.smithi087 on smithi087 2023-10-01T19:51:18.823 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:18.823 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:18.823 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:51:18.823 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:51:18.823 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:18.823 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:51:18.823 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:51:18.823 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:51:18.823 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:18.824 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:18.824 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:18 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:18.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:51:19.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T19:51:19.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi087.front.sepia.ceph.com:3000"}]: dispatch 2023-10-01T19:51:19.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: Deploying daemon agent.smithi057 on smithi057 2023-10-01T19:51:19.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: Deploying daemon agent.smithi087 on smithi087 2023-10-01T19:51:19.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:19.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:19.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:51:19.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:51:19.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:19.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:51:19.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:51:19.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:51:19.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:19.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:19.094 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T19:51:19.094 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi087.front.sepia.ceph.com:3000"}]: dispatch 2023-10-01T19:51:19.094 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: Deploying daemon agent.smithi057 on smithi057 2023-10-01T19:51:19.095 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: Deploying daemon agent.smithi087 on smithi087 2023-10-01T19:51:19.095 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:19.095 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:19.095 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:51:19.095 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:51:19.095 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:19.095 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:51:19.095 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:51:19.096 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:51:19.096 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:19.096 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:18 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:19.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:19 smithi087 ceph-mon[109386]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:20.046 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:19 smithi057 ceph-mon[108286]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:20.046 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:19 smithi057 ceph-mon[114331]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:21.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:51:20] "GET /metrics HTTP/1.1" 200 37879 "" "Prometheus/2.33.4" 2023-10-01T19:51:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:21 smithi087 ceph-mon[109386]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:21 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:51:21.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:21 smithi057 ceph-mon[108286]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:21.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:21.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:21.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:21 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:51:21.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:21 smithi057 ceph-mon[114331]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:21.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:21.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:21.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:21 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:51:23.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:22 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:22 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:22 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:22 smithi087 ceph-mon[109386]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:23.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:22 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:22 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:22 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:22 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:23.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[108286]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:23.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:23.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[114331]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:23.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:23.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:22 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:26.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:25 smithi087 ceph-mon[109386]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:26.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:25 smithi057 ceph-mon[108286]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:26.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:25 smithi057 ceph-mon[114331]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:27.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:26 smithi087 ceph-mon[109386]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:27.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:26 smithi057 ceph-mon[108286]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:27.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:26 smithi057 ceph-mon[114331]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:28.921 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:28 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:28.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:51:29.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:28 smithi057 ceph-mon[108286]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:29.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:28 smithi057 ceph-mon[114331]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:28 smithi087 ceph-mon[109386]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:31.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:30 smithi087 ceph-mon[109386]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:31.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:51:30] "GET /metrics HTTP/1.1" 200 37880 "" "Prometheus/2.33.4" 2023-10-01T19:51:31.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:30 smithi057 ceph-mon[108286]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:31.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:30 smithi057 ceph-mon[114331]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:33.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:32 smithi057 ceph-mon[108286]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:33.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:32 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:33.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:32 smithi057 ceph-mon[114331]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:33.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:32 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:32 smithi087 ceph-mon[109386]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:32 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:36.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:35 smithi087 ceph-mon[109386]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:36.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:35 smithi057 ceph-mon[108286]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:36.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:35 smithi057 ceph-mon[114331]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:36 smithi087 ceph-mon[109386]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:37.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:51:37.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:36 smithi057 ceph-mon[108286]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:37.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:51:37.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:36 smithi057 ceph-mon[114331]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:37.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:51:38.911 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:38 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:38.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:51:39.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:38 smithi057 ceph-mon[108286]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:39.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:38 smithi057 ceph-mon[114331]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:39.230 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:38 smithi087 ceph-mon[109386]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:40.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.510 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.510 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.510 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.510 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.510 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.510 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.617 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.617 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.617 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.617 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.618 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.618 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:51:40.882 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:51:40] "GET /metrics HTTP/1.1" 200 37877 "" "Prometheus/2.33.4" 2023-10-01T19:51:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:41 smithi087 ceph-mon[109386]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:41.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:41 smithi057 ceph-mon[108286]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:41.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:41 smithi057 ceph-mon[114331]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:43 smithi087 ceph-mon[109386]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:43 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:43.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:43 smithi057 ceph-mon[108286]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:43.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:43 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:43.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:43 smithi057 ceph-mon[114331]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:43.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:43 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:46.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:45 smithi087 ceph-mon[109386]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:46.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:45 smithi057 ceph-mon[108286]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:46.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:45 smithi057 ceph-mon[114331]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:47.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:46 smithi087 ceph-mon[109386]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:47.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:46 smithi057 ceph-mon[108286]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:47.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:46 smithi057 ceph-mon[114331]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:48.923 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:48 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:48.661Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:51:49.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:48 smithi087 ceph-mon[109386]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:49.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:48 smithi057 ceph-mon[108286]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:49.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:48 smithi057 ceph-mon[114331]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:51.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:50 smithi087 ceph-mon[109386]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:51.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:51:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:51:50] "GET /metrics HTTP/1.1" 200 37877 "" "Prometheus/2.33.4" 2023-10-01T19:51:51.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:50 smithi057 ceph-mon[108286]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:51.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:50 smithi057 ceph-mon[114331]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:52.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:51 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:51:52.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:51 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:51:52.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:51 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:51:53.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:52 smithi087 ceph-mon[109386]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:53.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:52 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:53.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:52 smithi057 ceph-mon[108286]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:53.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:52 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:53.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:52 smithi057 ceph-mon[114331]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:53.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:52 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:51:55.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:54 smithi087 ceph-mon[109386]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:55.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:54 smithi057 ceph-mon[108286]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:55.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:54 smithi057 ceph-mon[114331]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:57.162 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:56 smithi057 ceph-mon[108286]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:57.163 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:56 smithi057 ceph-mon[114331]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:57.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:56 smithi087 ceph-mon[109386]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:51:58.882 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:51:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:51:58.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:51:59.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:51:58 smithi057 ceph-mon[108286]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:51:58 smithi057 ceph-mon[114331]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:51:59.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:51:58 smithi087 ceph-mon[109386]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:00.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:00 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:00 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:00 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:00 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:00 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:00 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.689 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:00 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.689 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:00 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.690 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:00 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.690 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:00 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.690 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:00 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.690 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:00 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.756 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:00 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.756 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:00 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.756 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:00 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.757 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:00 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.757 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:00 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:00.757 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:00 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:01.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:52:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:52:00] "GET /metrics HTTP/1.1" 200 37877 "" "Prometheus/2.33.4" 2023-10-01T19:52:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:01 smithi087 ceph-mon[109386]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:01.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:01 smithi057 ceph-mon[108286]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:01.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:01 smithi057 ceph-mon[114331]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:03 smithi087 ceph-mon[109386]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:03 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:03.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:03 smithi057 ceph-mon[114331]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:03.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:03 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:03.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:03 smithi057 ceph-mon[108286]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:03.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:03 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:05.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:05 smithi087 ceph-mon[109386]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:05.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:05 smithi057 ceph-mon[108286]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:05.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:05 smithi057 ceph-mon[114331]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:06.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:06 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:52:06.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:06 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:52:06.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:06 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:52:07.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:07 smithi087 ceph-mon[109386]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:07.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:07 smithi057 ceph-mon[108286]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:07.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:07 smithi057 ceph-mon[114331]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:09.132 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:52:08 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:52:08.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:52:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:09 smithi087 ceph-mon[109386]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:09.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:09 smithi057 ceph-mon[108286]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:09.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:09 smithi057 ceph-mon[114331]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:11.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:10 smithi087 ceph-mon[109386]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:11.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:52:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:52:10] "GET /metrics HTTP/1.1" 200 37875 "" "Prometheus/2.33.4" 2023-10-01T19:52:11.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:10 smithi057 ceph-mon[108286]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:11.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:10 smithi057 ceph-mon[114331]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:12 smithi087 ceph-mon[109386]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:12 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:13.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:12 smithi057 ceph-mon[108286]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:13.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:12 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:13.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:12 smithi057 ceph-mon[114331]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:13.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:12 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:15.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:14 smithi087 ceph-mon[109386]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:14 smithi057 ceph-mon[108286]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:15.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:14 smithi057 ceph-mon[114331]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:16 smithi087 ceph-mon[109386]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:17.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:16 smithi057 ceph-mon[108286]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:17.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:16 smithi057 ceph-mon[114331]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:18.936 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:52:18 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:52:18.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:52:19.204 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:18 smithi087 ceph-mon[109386]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:19.204 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:18 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:52:19.204 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:18 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:52:19.204 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:18 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:52:19.204 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:18 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:19.272 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:18 smithi057 ceph-mon[114331]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:19.272 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:18 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:52:19.272 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:18 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:52:19.272 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:18 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:52:19.272 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:18 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:19.273 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:18 smithi057 ceph-mon[108286]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:19.273 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:18 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:52:19.273 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:18 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:52:19.273 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:18 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:52:19.273 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:18 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:20 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:20 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:20 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:20 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:20 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:20 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:20 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:20 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:20 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:20 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:20 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:20 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:52:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:52:20] "GET /metrics HTTP/1.1" 200 37875 "" "Prometheus/2.33.4" 2023-10-01T19:52:20.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:20 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:20 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:20 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:20 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:20 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:20.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:20 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:21 smithi087 ceph-mon[109386]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:21 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:52:21.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:21 smithi057 ceph-mon[114331]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:21.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:21 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:52:21.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:21 smithi057 ceph-mon[108286]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:21.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:21 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:52:23.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:23 smithi057 ceph-mon[108286]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:23.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:23 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:23 smithi057 ceph-mon[114331]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:23 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:24.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:23 smithi087 ceph-mon[109386]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:24.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:23 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:25.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:25 smithi057 ceph-mon[108286]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:25.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:25 smithi057 ceph-mon[114331]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:26.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:25 smithi087 ceph-mon[109386]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:27.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:27 smithi057 ceph-mon[108286]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:27.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:27 smithi057 ceph-mon[114331]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:28.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:27 smithi087 ceph-mon[109386]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:29.133 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:52:28 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:52:28.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:52:29.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:29 smithi057 ceph-mon[108286]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:29.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:29 smithi057 ceph-mon[114331]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:30.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:29 smithi087 ceph-mon[109386]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:31.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:30 smithi087 ceph-mon[109386]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:31.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:52:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:52:30] "GET /metrics HTTP/1.1" 200 37873 "" "Prometheus/2.33.4" 2023-10-01T19:52:31.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:30 smithi057 ceph-mon[108286]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:31.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:30 smithi057 ceph-mon[114331]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:32 smithi087 ceph-mon[109386]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:32 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:33.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:32 smithi057 ceph-mon[108286]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:33.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:32 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:32 smithi057 ceph-mon[114331]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:32 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:34 smithi087 ceph-mon[109386]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:35.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:34 smithi057 ceph-mon[108286]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:34 smithi057 ceph-mon[114331]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:37.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:36 smithi087 ceph-mon[109386]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:37.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:52:37.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:36 smithi057 ceph-mon[108286]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:37.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:52:37.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:36 smithi057 ceph-mon[114331]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:37.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:52:38.959 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:52:38 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:52:38.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:52:39.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:38 smithi057 ceph-mon[108286]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:39.237 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:38 smithi057 ceph-mon[114331]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:39.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:38 smithi087 ceph-mon[109386]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:40.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:40.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:40.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:40.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:40.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:40.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:40 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:40.884 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:52:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:52:40] "GET /metrics HTTP/1.1" 200 37868 "" "Prometheus/2.33.4" 2023-10-01T19:52:41.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:41.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:41.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:41.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:41.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:41.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:40 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:41.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:41.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:41.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:41.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:41.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:41.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:40 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:52:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:41 smithi087 ceph-mon[109386]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:42.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:41 smithi057 ceph-mon[108286]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:42.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:41 smithi057 ceph-mon[114331]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:44.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:43 smithi087 ceph-mon[109386]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:44.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:43 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:44.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:43 smithi057 ceph-mon[114331]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:44.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:43 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:44.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:43 smithi057 ceph-mon[108286]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:44.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:43 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:46.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:45 smithi087 ceph-mon[109386]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:46.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:45 smithi057 ceph-mon[108286]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:46.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:45 smithi057 ceph-mon[114331]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:46 smithi087 ceph-mon[109386]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:47.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:46 smithi057 ceph-mon[108286]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:47.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:46 smithi057 ceph-mon[114331]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:48.939 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:52:48 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:52:48.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:52:49.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:48 smithi087 ceph-mon[109386]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:49.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:48 smithi057 ceph-mon[108286]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:49.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:48 smithi057 ceph-mon[114331]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:51.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:50 smithi087 ceph-mon[109386]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:51.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:52:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:52:50] "GET /metrics HTTP/1.1" 200 37868 "" "Prometheus/2.33.4" 2023-10-01T19:52:51.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:50 smithi057 ceph-mon[108286]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:51.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:50 smithi057 ceph-mon[114331]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:52.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:51 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:52:52.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:51 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:52:52.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:51 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:52:53.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:52 smithi087 ceph-mon[109386]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:53.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:52 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:53.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:52 smithi057 ceph-mon[108286]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:53.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:52 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:53.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:52 smithi057 ceph-mon[114331]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:53.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:52 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:52:55.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:54 smithi087 ceph-mon[109386]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:55.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:54 smithi057 ceph-mon[108286]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:55.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:54 smithi057 ceph-mon[114331]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:57.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:56 smithi057 ceph-mon[108286]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:56 smithi057 ceph-mon[114331]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:57.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:56 smithi087 ceph-mon[109386]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:52:58.951 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:52:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:52:58.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:52:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:52:58 smithi087 ceph-mon[109386]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:59.392 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:52:58 smithi057 ceph-mon[108286]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:52:59.392 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:52:58 smithi057 ceph-mon[114331]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:01.106 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:53:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:53:00] "GET /metrics HTTP/1.1" 200 37883 "" "Prometheus/2.33.4" 2023-10-01T19:53:01.107 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:00 smithi087 ceph-mon[109386]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:01.107 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:00 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.107 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:00 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.107 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:00 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.107 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:00 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.107 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:00 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.107 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:00 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[108286]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[114331]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:01.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:01.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:00 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:03.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:02 smithi087 ceph-mon[109386]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:03.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:03 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:03.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:02 smithi057 ceph-mon[108286]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:03.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:02 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:03.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:03 smithi057 ceph-mon[114331]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:03.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:03 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:05.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:05 smithi087 ceph-mon[109386]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:05.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:05 smithi057 ceph-mon[108286]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:05.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:05 smithi057 ceph-mon[114331]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:06.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:06 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:53:06.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:06 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:53:06.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:06 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:53:07.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:07 smithi087 ceph-mon[109386]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:07.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:07 smithi057 ceph-mon[108286]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:07 smithi057 ceph-mon[114331]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:09.025 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:53:08 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:53:08.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:53:09.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:09 smithi087 ceph-mon[109386]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:09.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:09 smithi057 ceph-mon[108286]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:09.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:09 smithi057 ceph-mon[114331]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:11.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:10 smithi087 ceph-mon[109386]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:11.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:53:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:53:10] "GET /metrics HTTP/1.1" 200 37885 "" "Prometheus/2.33.4" 2023-10-01T19:53:11.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:10 smithi057 ceph-mon[108286]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:11.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:10 smithi057 ceph-mon[114331]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:12 smithi087 ceph-mon[109386]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:12 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:13.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:12 smithi057 ceph-mon[108286]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:13.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:12 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:13.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:12 smithi057 ceph-mon[114331]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:13.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:12 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:14 smithi087 ceph-mon[109386]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:14 smithi057 ceph-mon[108286]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:15.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:14 smithi057 ceph-mon[114331]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:16 smithi087 ceph-mon[109386]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:17.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:16 smithi057 ceph-mon[108286]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:17.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:16 smithi057 ceph-mon[114331]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:18.995 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:53:18 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:53:18.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:53:19.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:18 smithi087 ceph-mon[109386]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:19.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:18 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:53:19.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:18 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:53:19.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:18 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:53:19.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:18 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:19.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:18 smithi057 ceph-mon[108286]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:19.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:18 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:53:19.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:18 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:53:19.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:18 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:53:19.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:18 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:19.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:18 smithi057 ceph-mon[114331]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:19.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:18 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:53:19.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:18 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:53:19.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:18 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:53:19.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:18 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:53:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:53:20] "GET /metrics HTTP/1.1" 200 37885 "" "Prometheus/2.33.4" 2023-10-01T19:53:21.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[108286]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:21.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:53:21.395 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[114331]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:21.395 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.395 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.395 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.395 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.395 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.395 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.395 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:21 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:53:21.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:21 smithi087 ceph-mon[109386]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:21.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:21.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:21 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:53:23.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:23 smithi087 ceph-mon[109386]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:23.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:23 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:23.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:23 smithi057 ceph-mon[108286]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:23.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:23 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:23.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:23 smithi057 ceph-mon[114331]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:23.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:23 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:25.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:25 smithi087 ceph-mon[109386]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:25.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:25 smithi057 ceph-mon[108286]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:25 smithi057 ceph-mon[114331]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:27.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:27 smithi087 ceph-mon[109386]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:27.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:27 smithi057 ceph-mon[108286]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:27.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:27 smithi057 ceph-mon[114331]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:29.133 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:53:28 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:53:28.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:53:29.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:29 smithi087 ceph-mon[109386]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:29.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:29 smithi057 ceph-mon[108286]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:29.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:29 smithi057 ceph-mon[114331]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:31.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:30 smithi087 ceph-mon[109386]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:31.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:53:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:53:30] "GET /metrics HTTP/1.1" 200 37884 "" "Prometheus/2.33.4" 2023-10-01T19:53:31.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:30 smithi057 ceph-mon[108286]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:31.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:30 smithi057 ceph-mon[114331]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:32 smithi087 ceph-mon[109386]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:32 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:33.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:32 smithi057 ceph-mon[108286]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:33.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:32 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:32 smithi057 ceph-mon[114331]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:32 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:34 smithi087 ceph-mon[109386]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:35.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:34 smithi057 ceph-mon[108286]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:35.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:34 smithi057 ceph-mon[114331]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:37.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:36 smithi087 ceph-mon[109386]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:37.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:53:37.396 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:36 smithi057 ceph-mon[108286]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:37.396 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:53:37.397 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:36 smithi057 ceph-mon[114331]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:37.397 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:53:38.062 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch daemon redeploy "mgr.$(ceph mgr dump -f json | jq .standbys | jq .[] | jq -r .name)" --image quay.ceph.io/ceph-ci/ceph:$sha1' 2023-10-01T19:53:38.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:53:38 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:53:38.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:53:39.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:38 smithi057 ceph-mon[108286]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:39.120 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:38 smithi057 ceph-mon[114331]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:39.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:38 smithi087 ceph-mon[109386]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:40.188 INFO:teuthology.orchestra.run.smithi057.stdout:Scheduled to redeploy mgr.y on host 'smithi057' 2023-10-01T19:53:40.323 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:39 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2057159323' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-01T19:53:40.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:39 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2057159323' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-01T19:53:40.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:39 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2057159323' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-01T19:53:40.715 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch ps --refresh' 2023-10-01T19:53:41.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:53:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:53:40] "GET /metrics HTTP/1.1" 200 37881 "" "Prometheus/2.33.4" 2023-10-01T19:53:41.482 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:41.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: from='client.25027 -' entity='client.admin' cmd=[{"prefix": "orch daemon redeploy", "name": "mgr.y", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:53:41.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: Schedule redeploy daemon mgr.y 2023-10-01T19:53:41.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:53:41.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.483 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.484 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.484 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:41.484 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: from='client.25027 -' entity='client.admin' cmd=[{"prefix": "orch daemon redeploy", "name": "mgr.y", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:53:41.484 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.484 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: Schedule redeploy daemon mgr.y 2023-10-01T19:53:41.484 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:53:41.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: from='client.25027 -' entity='client.admin' cmd=[{"prefix": "orch daemon redeploy", "name": "mgr.y", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:53:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: Schedule redeploy daemon mgr.y 2023-10-01T19:53:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:53:41.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:41.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:43.061 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T19:53:43.061 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 2s ago 14m - - 2023-10-01T19:53:43.062 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 2s ago 13m - - 2023-10-01T19:53:43.062 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (2m) 2s ago 10m 24.9M - 0.23.0 ba2b418f427c 0b16afb678d9 2023-10-01T19:53:43.062 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (2m) 2s ago 10m 48.5M - 8.3.5 dad864ee21e9 ca8ad8b1ccf8 2023-10-01T19:53:43.062 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (2m) 2s ago 9m 56.7M - 3.5 e1d6a67b021e d11263431bdb 2023-10-01T19:53:43.062 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283 running (5m) 2s ago 13m 520M - 18.0.0-6505-g37d71fd3 bea6d6056140 f7b2c168ac86 2023-10-01T19:53:43.062 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:9283 running (14m) 2s ago 14m 431M - 17.2.0 e1d6a67b021e d546d6fb718a 2023-10-01T19:53:43.062 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (14m) 2s ago 15m 133M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T19:53:43.062 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (13m) 2s ago 13m 92.3M 2048M 17.2.0 e1d6a67b021e 4734e639b775 2023-10-01T19:53:43.062 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (13m) 2s ago 13m 85.0M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T19:53:43.062 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (3m) 2s ago 10m 23.9M - 1.3.1 1dbe0e931976 41af659015b7 2023-10-01T19:53:43.062 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (2m) 2s ago 10m 24.1M - 1.3.1 1dbe0e931976 dd38d7fd60ea 2023-10-01T19:53:43.063 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (13m) 2s ago 13m 116M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T19:53:43.063 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (12m) 2s ago 12m 122M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T19:53:43.063 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (12m) 2s ago 12m 94.1M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T19:53:43.063 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (12m) 2s ago 12m 126M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T19:53:43.063 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (12m) 2s ago 12m 123M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T19:53:43.063 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (11m) 2s ago 11m 120M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T19:53:43.063 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (11m) 2s ago 11m 98.7M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T19:53:43.063 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (11m) 2s ago 11m 118M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T19:53:43.063 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (2m) 2s ago 10m 68.7M - 2.33.4 514e6a882f6e f7dd35d9dddb 2023-10-01T19:53:43.063 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (9m) 2s ago 9m 96.4M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T19:53:43.063 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (9m) 2s ago 9m 92.2M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T19:53:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:43 smithi087 ceph-mon[109386]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:43 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:43 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:43 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:53:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:43 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:53:43.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:43 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:43.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:43 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:53:43.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:43 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:53:43.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:43 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T19:53:43.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:43 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:53:43.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:43 smithi087 ceph-mon[109386]: Deploying daemon mgr.y on smithi057 2023-10-01T19:53:43.634 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[114331]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:43.635 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:43.635 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:43.635 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:53:43.635 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:53:43.635 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:43.635 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:53:43.635 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:53:43.635 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T19:53:43.635 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:53:43.635 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[114331]: Deploying daemon mgr.y on smithi057 2023-10-01T19:53:43.635 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[108286]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:43.636 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:43.636 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:43.636 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:53:43.636 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:53:43.636 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:43.636 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:53:43.636 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T19:53:43.636 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T19:53:43.636 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:53:43.636 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:43 smithi057 ceph-mon[108286]: Deploying daemon mgr.y on smithi057 2023-10-01T19:53:43.747 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'sleep 180' 2023-10-01T19:53:43.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:43 smithi057 systemd[1]: Stopping Ceph mgr.y for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:53:44.448 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:44 smithi057 bash[161647]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y 2023-10-01T19:53:44.448 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:44 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-10-01T19:53:44.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:44 smithi057 ceph-mon[108286]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch ps", "refresh": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:53:44.830 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:44 smithi057 ceph-mon[114331]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch ps", "refresh": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:53:44.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:44 smithi087 ceph-mon[109386]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch ps", "refresh": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:53:45.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:44 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.y.service: Failed with result 'exit-code'. 2023-10-01T19:53:45.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:44 smithi057 systemd[1]: Stopped Ceph mgr.y for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:53:45.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:45 smithi057 systemd[1]: Starting Ceph mgr.y for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:53:45.552 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:45 smithi057 ceph-mon[108286]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:45.553 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:45 smithi057 ceph-mon[114331]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:45.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:45 smithi087 ceph-mon[109386]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:45.911 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:45 smithi057 ceph-mgr[161953]: -- 172.21.15.57:0/3503002445 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563f95aa91e0 con 0x563f95a7ec00 2023-10-01T19:53:45.911 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:45 smithi057 bash[161888]: 50d5f27e09f07241ccb5406c57cd8e76f183cfa2ab8769a1c36f72d40b371f38 2023-10-01T19:53:45.911 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:45 smithi057 systemd[1]: Started Ceph mgr.y for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:53:45.911 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:45 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:45.706+0000 7f8715bcb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T19:53:45.911 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:45 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:45.910+0000 7f8715bcb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T19:53:46.645 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:46.285+0000 7f8715bcb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T19:53:46.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:46.870+0000 7f8715bcb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T19:53:46.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:46.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:46.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:46.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:46.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:53:46.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:53:46.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:53:46.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:46.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:46.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:46.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:46.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:46.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:53:46.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:53:46.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:53:46.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:46 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:46 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:46 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:46 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:46 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:46 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:53:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:46 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:53:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:46 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:53:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:46 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:53:47.422 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:47 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:47.129+0000 7f8715bcb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-01T19:53:47.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:47 smithi057 ceph-mon[108286]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:47.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:47 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:47.614+0000 7f8715bcb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T19:53:47.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:47 smithi057 ceph-mon[114331]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:48.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:47 smithi087 ceph-mon[109386]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:48.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:48 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:48.435+0000 7f8715bcb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T19:53:48.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:48 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:48.535+0000 7f8715bcb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T19:53:49.133 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:53:48 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:53:48.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:53:49.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:49.237+0000 7f8715bcb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T19:53:49.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:49.336+0000 7f8715bcb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T19:53:49.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:49.438+0000 7f8715bcb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T19:53:50.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:49 smithi087 ceph-mon[109386]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:50.167 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:49 smithi057 ceph-mon[108286]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:50.167 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:49.701+0000 7f8715bcb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T19:53:50.168 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:49.809+0000 7f8715bcb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T19:53:50.168 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:49.919+0000 7f8715bcb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T19:53:50.168 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:49 smithi057 ceph-mon[114331]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:50.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:50 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:50.167+0000 7f8715bcb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T19:53:50.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:50 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:50.279+0000 7f8715bcb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T19:53:50.864 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:50 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:50.494+0000 7f8715bcb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T19:53:50.865 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:50 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:50.595+0000 7f8715bcb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T19:53:51.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:50 smithi087 ceph-mon[109386]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:51.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:53:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:53:50] "GET /metrics HTTP/1.1" 200 37881 "" "Prometheus/2.33.4" 2023-10-01T19:53:51.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:50 smithi057 ceph-mon[108286]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:51.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:50 smithi057 ceph-mon[114331]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:51.868 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:51 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:51.481+0000 7f8715bcb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T19:53:51.868 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:51 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:51.710+0000 7f8715bcb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T19:53:52.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:51 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:53:52.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:51 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:53:52.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:51 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:51.922+0000 7f8715bcb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:53:52.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:51 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:53:52.885 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:52 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:52.690+0000 7f8715bcb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T19:53:53.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:52 smithi057 ceph-mon[108286]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:53.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:52 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:53.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:52 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:52.905+0000 7f8715bcb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:53:53.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:53.120+0000 7f8715bcb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T19:53:53.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:52 smithi057 ceph-mon[114331]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:53.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:52 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:53.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:52 smithi087 ceph-mon[109386]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:53:53.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:52 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:53:53.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:53.256+0000 7f8715bcb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T19:53:53.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:19:53:53] ENGINE Bus STARTING 2023-10-01T19:53:53.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: CherryPy Checker: 2023-10-01T19:53:53.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: The Application mounted at '' has an empty config. 2023-10-01T19:53:53.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T19:53:53.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:19:53:53] ENGINE Serving on http://:::9283 2023-10-01T19:53:53.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:19:53:53] ENGINE Bus STARTED 2023-10-01T19:53:54.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-mon[108286]: Standby manager daemon y restarted 2023-10-01T19:53:54.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-mon[108286]: Standby manager daemon y started 2023-10-01T19:53:54.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-01T19:53:54.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:53:54.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-01T19:53:54.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:53:54.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-mon[114331]: Standby manager daemon y restarted 2023-10-01T19:53:54.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-mon[114331]: Standby manager daemon y started 2023-10-01T19:53:54.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-01T19:53:54.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:53:54.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-01T19:53:54.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:53 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:53:54.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:53 smithi087 ceph-mon[109386]: Standby manager daemon y restarted 2023-10-01T19:53:54.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:53 smithi087 ceph-mon[109386]: Standby manager daemon y started 2023-10-01T19:53:54.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:53 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-01T19:53:54.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:53 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:53:54.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:53 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-01T19:53:54.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:53 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:53:55.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:54 smithi057 ceph-mon[108286]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:55.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:54 smithi057 ceph-mon[108286]: mgrmap e28: x(active, since 3m), standbys: y 2023-10-01T19:53:55.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:54 smithi057 ceph-mon[114331]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:55.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:54 smithi057 ceph-mon[114331]: mgrmap e28: x(active, since 3m), standbys: y 2023-10-01T19:53:55.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:54 smithi087 ceph-mon[109386]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:55.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:54 smithi087 ceph-mon[109386]: mgrmap e28: x(active, since 3m), standbys: y 2023-10-01T19:53:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:56 smithi087 ceph-mon[109386]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:56 smithi057 ceph-mon[108286]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:57.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:56 smithi057 ceph-mon[114331]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:53:58.938 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:53:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:53:58.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:53:59.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:53:58 smithi057 ceph-mon[108286]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 82 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T19:53:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:53:58 smithi057 ceph-mon[114331]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 82 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T19:53:59.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:53:58 smithi087 ceph-mon[109386]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 82 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T19:54:00.882 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:54:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:54:00] "GET /metrics HTTP/1.1" 200 37885 "" "Prometheus/2.33.4" 2023-10-01T19:54:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:01 smithi087 ceph-mon[109386]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 82 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T19:54:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[108286]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 82 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T19:54:01.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[114331]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 82 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T19:54:01.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:01.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:03.857 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:03 smithi087 ceph-mon[109386]: pgmap v113: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T19:54:03.857 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:03 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:03.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:03 smithi057 ceph-mon[108286]: pgmap v113: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T19:54:03.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:03 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:03.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:03 smithi057 ceph-mon[114331]: pgmap v113: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T19:54:03.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:03 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:05 smithi087 ceph-mon[109386]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T19:54:05.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:05 smithi057 ceph-mon[108286]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T19:54:05.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:05 smithi057 ceph-mon[114331]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T19:54:06.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:06 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:54:06.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:06 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:54:06.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:06 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:54:07.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:07 smithi087 ceph-mon[109386]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T19:54:07.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:07 smithi057 ceph-mon[108286]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T19:54:07.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:07 smithi057 ceph-mon[114331]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T19:54:09.133 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:54:08 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:54:08.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:54:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:09 smithi087 ceph-mon[109386]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T19:54:09.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:09 smithi057 ceph-mon[108286]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T19:54:09.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:09 smithi057 ceph-mon[114331]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T19:54:11.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:10 smithi087 ceph-mon[109386]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:11.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:54:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:54:10] "GET /metrics HTTP/1.1" 200 37882 "" "Prometheus/2.33.4" 2023-10-01T19:54:11.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:10 smithi057 ceph-mon[108286]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:11.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:10 smithi057 ceph-mon[114331]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:12 smithi087 ceph-mon[109386]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:12 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:13.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:12 smithi057 ceph-mon[114331]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:13.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:12 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:13.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:12 smithi057 ceph-mon[108286]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:13.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:12 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:14 smithi087 ceph-mon[109386]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:14 smithi057 ceph-mon[108286]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:15.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:14 smithi057 ceph-mon[114331]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:16 smithi087 ceph-mon[109386]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:17.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:16 smithi057 ceph-mon[108286]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:17.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:16 smithi057 ceph-mon[114331]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:18.981 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:54:18 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:54:18.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:54:19.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:18 smithi087 ceph-mon[109386]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:19.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:18 smithi057 ceph-mon[108286]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:19.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:18 smithi057 ceph-mon[114331]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:21.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:54:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:54:20] "GET /metrics HTTP/1.1" 200 37882 "" "Prometheus/2.33.4" 2023-10-01T19:54:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:21 smithi087 ceph-mon[109386]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:21 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:54:21.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[114331]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:21.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:54:21.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[108286]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:21 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:54:23.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:23 smithi057 ceph-mon[108286]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:23.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:23 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:23.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:23 smithi057 ceph-mon[114331]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:23.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:23 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:23 smithi087 ceph-mon[109386]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:23 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:25.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:25 smithi087 ceph-mon[109386]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:25.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:25 smithi057 ceph-mon[108286]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:25.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:25 smithi057 ceph-mon[114331]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:27.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:27 smithi087 ceph-mon[109386]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:27.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:27 smithi057 ceph-mon[108286]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:27.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:27 smithi057 ceph-mon[114331]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:29.028 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:54:28 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:54:28.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:54:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:29 smithi087 ceph-mon[109386]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:29.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:29 smithi057 ceph-mon[108286]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:29.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:29 smithi057 ceph-mon[114331]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:31.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:30 smithi087 ceph-mon[109386]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:31.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:54:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:54:30] "GET /metrics HTTP/1.1" 200 37887 "" "Prometheus/2.33.4" 2023-10-01T19:54:31.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:30 smithi057 ceph-mon[108286]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:31.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:30 smithi057 ceph-mon[114331]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:32 smithi087 ceph-mon[109386]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:33.455 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:32 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:33.456 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:32 smithi057 ceph-mon[108286]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:33.456 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:32 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:33.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:32 smithi057 ceph-mon[114331]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:33.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:32 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:34 smithi087 ceph-mon[109386]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:35.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:34 smithi057 ceph-mon[108286]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:34 smithi057 ceph-mon[114331]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:36.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:35 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:54:36.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:35 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:54:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:35 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:54:37.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:37 smithi087 ceph-mon[109386]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:37.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:37 smithi057 ceph-mon[108286]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:37.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:37 smithi057 ceph-mon[114331]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:39.013 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:54:38 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:54:38.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:54:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:39 smithi087 ceph-mon[109386]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:39.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:39 smithi057 ceph-mon[108286]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:39.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:39 smithi057 ceph-mon[114331]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:41.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:54:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:54:40] "GET /metrics HTTP/1.1" 200 37889 "" "Prometheus/2.33.4" 2023-10-01T19:54:41.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[108286]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:41.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[114331]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:41 smithi087 ceph-mon[109386]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:41.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:43 smithi087 ceph-mon[109386]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:43.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:43 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:43.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:43 smithi057 ceph-mon[108286]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:43.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:43 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:43.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:43 smithi057 ceph-mon[114331]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:43.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:43 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:45.545 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:45 smithi087 ceph-mon[109386]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:45.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:45 smithi057 ceph-mon[108286]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:45.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:45 smithi057 ceph-mon[114331]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:46 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:54:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:46 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:54:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:46 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:54:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:46 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:46.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:46 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:54:46.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:46 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:54:46.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:46 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:54:46.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:46 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:46.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:46 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:54:46.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:46 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:54:46.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:46 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:54:46.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:46 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:54:48.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:47 smithi087 ceph-mon[109386]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:48.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:47 smithi057 ceph-mon[108286]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:48.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:47 smithi057 ceph-mon[114331]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:49.133 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:54:48 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:54:48.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:54:50.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:49 smithi087 ceph-mon[109386]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:50.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:49 smithi057 ceph-mon[108286]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:50.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:49 smithi057 ceph-mon[114331]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:51.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:50 smithi087 ceph-mon[109386]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:51.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:54:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:54:50] "GET /metrics HTTP/1.1" 200 37889 "" "Prometheus/2.33.4" 2023-10-01T19:54:51.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:50 smithi057 ceph-mon[108286]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:51.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:50 smithi057 ceph-mon[114331]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:52.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:51 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:54:52.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:51 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:54:52.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:51 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:54:53.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:52 smithi057 ceph-mon[108286]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:53.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:52 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:53.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:52 smithi057 ceph-mon[114331]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:53.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:52 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:53.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:52 smithi087 ceph-mon[109386]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:53.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:52 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:54:55.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:54 smithi087 ceph-mon[109386]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:55.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:54 smithi057 ceph-mon[108286]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:55.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:54 smithi057 ceph-mon[114331]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:56 smithi087 ceph-mon[109386]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:56 smithi057 ceph-mon[108286]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:57.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:56 smithi057 ceph-mon[114331]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:54:58.987 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:54:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:54:58.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:54:59.330 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:54:58 smithi087 ceph-mon[109386]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:59.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:54:58 smithi057 ceph-mon[108286]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:54:59.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:54:58 smithi057 ceph-mon[114331]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:01.076 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:55:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:55:00] "GET /metrics HTTP/1.1" 200 37886 "" "Prometheus/2.33.4" 2023-10-01T19:55:01.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:01 smithi087 ceph-mon[109386]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:01.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[108286]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:01.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[114331]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:01.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:01.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:03 smithi087 ceph-mon[109386]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:03 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:03.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:03 smithi057 ceph-mon[108286]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:03 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:03 smithi057 ceph-mon[114331]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:03 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:05.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:05 smithi057 ceph-mon[108286]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:05.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:05 smithi057 ceph-mon[114331]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:05.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:05 smithi087 ceph-mon[109386]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:06.591 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:06 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:55:06.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:06 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:55:06.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:06 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:55:07.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:07 smithi057 ceph-mon[108286]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:07.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:07 smithi057 ceph-mon[114331]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:07.721 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:07 smithi087 ceph-mon[109386]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:09.133 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:55:08 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:55:08.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:55:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:09 smithi087 ceph-mon[109386]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:09.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:09 smithi057 ceph-mon[108286]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:09.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:09 smithi057 ceph-mon[114331]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:11.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:55:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:55:10] "GET /metrics HTTP/1.1" 200 37887 "" "Prometheus/2.33.4" 2023-10-01T19:55:11.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:10 smithi087 ceph-mon[109386]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:11.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:10 smithi057 ceph-mon[108286]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:11.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:10 smithi057 ceph-mon[114331]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:12 smithi087 ceph-mon[109386]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:12 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:13.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:12 smithi057 ceph-mon[108286]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:13.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:12 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:13.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:12 smithi057 ceph-mon[114331]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:13.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:12 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:14 smithi087 ceph-mon[109386]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:15.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:14 smithi057 ceph-mon[108286]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:15.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:14 smithi057 ceph-mon[114331]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:17.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:17 smithi087 ceph-mon[109386]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:17.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:17 smithi057 ceph-mon[108286]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:17.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:17 smithi057 ceph-mon[114331]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:19.010 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:55:18 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:55:18.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:55:19.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:19 smithi087 ceph-mon[109386]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:19.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:19 smithi057 ceph-mon[108286]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:19.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:19 smithi057 ceph-mon[114331]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:21.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:55:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:55:20] "GET /metrics HTTP/1.1" 200 37887 "" "Prometheus/2.33.4" 2023-10-01T19:55:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:21 smithi087 ceph-mon[109386]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:21 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:55:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[108286]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:21.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:55:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[114331]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:21.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:55:21.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:21.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:23 smithi087 ceph-mon[109386]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:23 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:23.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:23 smithi057 ceph-mon[108286]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:23.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:23 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:23 smithi057 ceph-mon[114331]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:23 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:25.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:25 smithi087 ceph-mon[109386]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:25.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:25 smithi057 ceph-mon[108286]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:25.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:25 smithi057 ceph-mon[114331]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:27.845 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:27 smithi087 ceph-mon[109386]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:27.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:27 smithi057 ceph-mon[108286]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:27.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:27 smithi057 ceph-mon[114331]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:29.133 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:55:28 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:55:28.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:55:29.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:29 smithi087 ceph-mon[109386]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:29.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:29 smithi057 ceph-mon[108286]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:29.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:29 smithi057 ceph-mon[114331]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:31.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:30 smithi087 ceph-mon[109386]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:31.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:55:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:55:30] "GET /metrics HTTP/1.1" 200 37881 "" "Prometheus/2.33.4" 2023-10-01T19:55:31.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:30 smithi057 ceph-mon[108286]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:31.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:30 smithi057 ceph-mon[114331]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:32 smithi087 ceph-mon[109386]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:32 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:33.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:32 smithi057 ceph-mon[108286]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:33.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:32 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:32 smithi057 ceph-mon[114331]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:32 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:34 smithi087 ceph-mon[109386]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:35.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:34 smithi057 ceph-mon[108286]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:34 smithi057 ceph-mon[114331]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:36.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:55:36.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:55:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:55:37.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:37 smithi087 ceph-mon[109386]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:37.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:37 smithi057 ceph-mon[108286]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:37.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:37 smithi057 ceph-mon[114331]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:39.022 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:55:38 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:55:38.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:55:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:39 smithi087 ceph-mon[109386]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:39.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:39 smithi057 ceph-mon[108286]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:39.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:39 smithi057 ceph-mon[114331]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:41.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:55:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:55:40] "GET /metrics HTTP/1.1" 200 37888 "" "Prometheus/2.33.4" 2023-10-01T19:55:41.951 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[108286]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:41.951 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:41.951 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:41.952 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:41.952 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:41.952 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:41.953 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:41.953 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[114331]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:41.953 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:41.953 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:41.954 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:41.954 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:41.954 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:41.954 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:41 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:41 smithi087 ceph-mon[109386]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:41 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:44.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:43 smithi087 ceph-mon[109386]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:44.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:43 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:44.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:43 smithi057 ceph-mon[108286]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:44.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:43 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:44.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:43 smithi057 ceph-mon[114331]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:44.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:43 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:46.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:45 smithi087 ceph-mon[109386]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:45 smithi057 ceph-mon[108286]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:46.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:45 smithi057 ceph-mon[114331]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:46 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:55:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:46 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:55:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:46 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:55:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:46 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:47.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:46 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:55:47.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:46 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:55:47.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:46 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:55:47.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:46 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:47.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:46 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:55:47.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:46 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:55:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:46 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:55:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:46 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:55:48.113 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:47 smithi087 ceph-mon[109386]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:48.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:47 smithi057 ceph-mon[108286]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:48.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:47 smithi057 ceph-mon[114331]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:48.792 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:55:48 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:55:48.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:55:49.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:49 smithi087 ceph-mon[109386]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:50.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:49 smithi057 ceph-mon[108286]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:50.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:49 smithi057 ceph-mon[114331]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:51.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:50 smithi087 ceph-mon[109386]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:51.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:55:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:55:50] "GET /metrics HTTP/1.1" 200 37888 "" "Prometheus/2.33.4" 2023-10-01T19:55:51.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:50 smithi057 ceph-mon[108286]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:51.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:50 smithi057 ceph-mon[114331]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:52.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:51 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:55:52.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:51 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:55:52.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:51 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:55:53.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:52 smithi057 ceph-mon[108286]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:53.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:52 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:53.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:52 smithi057 ceph-mon[114331]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:53.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:52 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:53.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:52 smithi087 ceph-mon[109386]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:53.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:52 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:55:55.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:54 smithi087 ceph-mon[109386]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:55.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:54 smithi057 ceph-mon[108286]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:55.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:54 smithi057 ceph-mon[114331]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:57 smithi087 ceph-mon[109386]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:57 smithi057 ceph-mon[108286]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:57.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:57 smithi057 ceph-mon[114331]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:55:59.019 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:55:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:55:58.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"smithi087:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.57\", device_class=\"ssd\", hostname=\"smithi057\", instance=\"172.21.15.87:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.57\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2023-10-01T19:55:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:55:59 smithi087 ceph-mon[109386]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:59.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:55:59 smithi057 ceph-mon[108286]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:55:59.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:55:59 smithi057 ceph-mon[114331]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:01.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:56:00] "GET /metrics HTTP/1.1" 200 37886 "" "Prometheus/2.33.4" 2023-10-01T19:56:02.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:01 smithi087 ceph-mon[109386]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:02.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:01 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[108286]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:02.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[114331]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:02.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:02.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:01 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:04.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:03 smithi087 ceph-mon[109386]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:04.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:03 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:04.152 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:03 smithi057 ceph-mon[108286]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:04.153 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:03 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:04.153 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:03 smithi057 ceph-mon[114331]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:04.153 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:03 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:06.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:05 smithi087 ceph-mon[109386]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:06.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:05 smithi057 ceph-mon[108286]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:06.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:05 smithi057 ceph-mon[114331]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:07.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:06 smithi057 ceph-mon[108286]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:07.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:06 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:07.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:06 smithi057 ceph-mon[114331]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:07.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:06 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:07.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:06 smithi087 ceph-mon[109386]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:07.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:06 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:09.323 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:09 smithi087 ceph-mon[109386]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:09.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:09 smithi057 ceph-mon[108286]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:09.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:09 smithi057 ceph-mon[114331]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:11.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:10 smithi087 ceph-mon[109386]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:11.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:56:10] "GET /metrics HTTP/1.1" 200 37881 "" "Prometheus/2.33.4" 2023-10-01T19:56:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:10 smithi057 ceph-mon[108286]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:11.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:10 smithi057 ceph-mon[114331]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:13 smithi087 ceph-mon[109386]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:13 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:13.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:13 smithi057 ceph-mon[108286]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:13.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:13 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:13.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:13 smithi057 ceph-mon[114331]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:13.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:13 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:15 smithi087 ceph-mon[109386]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:15 smithi057 ceph-mon[108286]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:15.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:15 smithi057 ceph-mon[114331]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:17 smithi087 ceph-mon[109386]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:17.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:17 smithi057 ceph-mon[108286]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:17.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:17 smithi057 ceph-mon[114331]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:19.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:19 smithi087 ceph-mon[109386]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:19.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:19 smithi057 ceph-mon[108286]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:19.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:19 smithi057 ceph-mon[114331]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:20 smithi087 ceph-mon[109386]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:21.134 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:56:20] "GET /metrics HTTP/1.1" 200 37881 "" "Prometheus/2.33.4" 2023-10-01T19:56:21.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:20 smithi057 ceph-mon[108286]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:21.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:20 smithi057 ceph-mon[114331]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:22.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:21 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:22.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:21 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:22.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:22.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:22.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:21 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:23.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:22 smithi057 ceph-mon[108286]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:23.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:22 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:23.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:22 smithi057 ceph-mon[114331]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:23.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:22 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:22 smithi087 ceph-mon[109386]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:22 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:25.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:25 smithi087 ceph-mon[109386]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:25.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:25 smithi057 ceph-mon[108286]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:25.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:25 smithi057 ceph-mon[114331]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:27.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:27 smithi087 ceph-mon[109386]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:27.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:27 smithi057 ceph-mon[108286]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:27.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:27 smithi057 ceph-mon[114331]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:29 smithi087 ceph-mon[109386]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:29.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:29 smithi057 ceph-mon[108286]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:29.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:29 smithi057 ceph-mon[114331]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:30.923 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:56:30] "GET /metrics HTTP/1.1" 200 37882 "" "Prometheus/2.33.4" 2023-10-01T19:56:31.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:30 smithi087 ceph-mon[109386]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:31.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:30 smithi057 ceph-mon[108286]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:31.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:30 smithi057 ceph-mon[114331]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:33 smithi087 ceph-mon[109386]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:33 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:33.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:33 smithi057 ceph-mon[108286]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:33.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:33 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:33 smithi057 ceph-mon[114331]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:33 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:35 smithi087 ceph-mon[109386]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:35.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:35 smithi057 ceph-mon[108286]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:35 smithi057 ceph-mon[114331]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:36.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:36 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:36.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:36 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:36 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:37.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:37 smithi087 ceph-mon[109386]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:37.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:37 smithi057 ceph-mon[114331]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:37.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:37 smithi057 ceph-mon[108286]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:39 smithi087 ceph-mon[109386]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:39.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:39 smithi057 ceph-mon[108286]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:39.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:39 smithi057 ceph-mon[114331]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:41.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:40 smithi087 ceph-mon[109386]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:41.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:56:40] "GET /metrics HTTP/1.1" 200 37887 "" "Prometheus/2.33.4" 2023-10-01T19:56:41.200 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:40 smithi057 ceph-mon[108286]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:41.200 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:40 smithi057 ceph-mon[114331]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:42.612 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:42 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.612 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:42 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.612 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:42 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.612 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:42 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.612 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:42 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.612 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:42 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.613 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:42 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.613 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:42 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.613 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:42 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.613 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:42 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.613 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:42 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.613 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:42 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:42 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:42 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:42 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:42 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:42 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:42.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:42 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:44.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:43 smithi087 ceph-mon[109386]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:44.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:43 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:44.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:43 smithi057 ceph-mon[108286]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:44.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:43 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:44.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:43 smithi057 ceph-mon[114331]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:44.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:43 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:45.899 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch ps' 2023-10-01T19:56:46.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:45 smithi087 ceph-mon[109386]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:46.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:45 smithi057 ceph-mon[108286]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:46.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:45 smithi057 ceph-mon[114331]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:46.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:46 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:56:46.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:46 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:56:46.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:46 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:56:46.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:46 smithi057 ceph-mon[108286]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:46.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:46 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:56:46.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:46 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:56:46.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:46 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:56:46.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:46 smithi057 ceph-mon[114331]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:46 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:56:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:46 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:56:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:46 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:56:47.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:46 smithi087 ceph-mon[109386]: from='mgr.24908 ' entity='mgr.x' 2023-10-01T19:56:47.566 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T19:56:47.567 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 6s ago 17m - - 2023-10-01T19:56:47.567 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 5s ago 17m - - 2023-10-01T19:56:47.567 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (5m) 6s ago 13m 27.2M - 0.23.0 ba2b418f427c 0b16afb678d9 2023-10-01T19:56:47.567 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (5m) 5s ago 13m 57.5M - 8.3.5 dad864ee21e9 ca8ad8b1ccf8 2023-10-01T19:56:47.567 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (5m) 6s ago 12m 57.5M - 3.5 e1d6a67b021e d11263431bdb 2023-10-01T19:56:47.567 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283 running (8m) 5s ago 16m 529M - 18.0.0-6505-g37d71fd3 bea6d6056140 f7b2c168ac86 2023-10-01T19:56:47.567 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (3m) 6s ago 18m 449M - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T19:56:47.567 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (18m) 6s ago 18m 143M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T19:56:47.567 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (16m) 5s ago 16m 90.9M 2048M 17.2.0 e1d6a67b021e 4734e639b775 2023-10-01T19:56:47.567 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (16m) 6s ago 16m 80.9M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (6m) 6s ago 13m 24.0M - 1.3.1 1dbe0e931976 41af659015b7 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (5m) 5s ago 13m 28.4M - 1.3.1 1dbe0e931976 dd38d7fd60ea 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (16m) 6s ago 16m 131M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (16m) 6s ago 16m 138M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (15m) 6s ago 15m 103M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (15m) 6s ago 15m 147M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (15m) 5s ago 15m 142M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (14m) 5s ago 14m 137M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (14m) 5s ago 14m 112M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (14m) 5s ago 14m 135M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (5m) 5s ago 13m 69.2M - 2.33.4 514e6a882f6e f7dd35d9dddb 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (12m) 6s ago 12m 97.2M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T19:56:47.568 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (12m) 5s ago 12m 93.2M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T19:56:47.956 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions' 2023-10-01T19:56:48.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:47 smithi087 ceph-mon[109386]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:48.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:47 smithi057 ceph-mon[108286]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:48.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:47 smithi057 ceph-mon[114331]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:49.077 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:48 smithi057 ceph-mon[108286]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:56:49.078 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:48 smithi057 ceph-mon[114331]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:56:49.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:48 smithi087 ceph-mon[109386]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T19:56:49.598 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T19:56:49.598 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2023-10-01T19:56:49.598 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2023-10-01T19:56:49.599 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:56:49.599 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2023-10-01T19:56:49.599 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T19:56:49.599 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:56:49.599 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2023-10-01T19:56:49.599 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2023-10-01T19:56:49.599 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:56:49.599 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": {}, 2023-10-01T19:56:49.599 INFO:teuthology.orchestra.run.smithi057.stdout: "rgw": { 2023-10-01T19:56:49.599 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T19:56:49.600 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T19:56:49.600 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2023-10-01T19:56:49.600 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 13, 2023-10-01T19:56:49.600 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T19:56:49.600 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T19:56:49.600 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T19:56:50.021 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph health detail' 2023-10-01T19:56:50.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:49 smithi057 ceph-mon[108286]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:50.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:49 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2109858844' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T19:56:50.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:49 smithi057 ceph-mon[114331]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:50.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:49 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2109858844' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T19:56:50.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:49 smithi087 ceph-mon[109386]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:50.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:49 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2109858844' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T19:56:51.093 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:50 smithi087 ceph-mon[109386]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:51.093 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:56:50] "GET /metrics HTTP/1.1" 200 37887 "" "Prometheus/2.33.4" 2023-10-01T19:56:51.165 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:50 smithi057 ceph-mon[108286]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:51.165 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:50 smithi057 ceph-mon[114331]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:51.675 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2023-10-01T19:56:52.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:51 smithi057 ceph-mon[108286]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:52.040 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:51 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1587813527' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T19:56:52.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:51 smithi057 ceph-mon[114331]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:52.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:51 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1587813527' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T19:56:52.089 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph -s' 2023-10-01T19:56:52.356 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:51 smithi087 ceph-mon[109386]: from='mgr.24908 172.21.15.87:0/3413464314' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:52.356 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:51 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1587813527' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T19:56:53.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:52 smithi057 ceph-mon[108286]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:53.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:52 smithi057 ceph-mon[108286]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:53.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:52 smithi057 ceph-mon[114331]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:53.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:52 smithi057 ceph-mon[114331]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:53.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:52 smithi087 ceph-mon[109386]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:56:53.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:52 smithi087 ceph-mon[109386]: from='client.15066 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:56:53.700 INFO:teuthology.orchestra.run.smithi057.stdout: cluster: 2023-10-01T19:56:53.700 INFO:teuthology.orchestra.run.smithi057.stdout: id: 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T19:56:53.700 INFO:teuthology.orchestra.run.smithi057.stdout: health: HEALTH_OK 2023-10-01T19:56:53.700 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:56:53.701 INFO:teuthology.orchestra.run.smithi057.stdout: services: 2023-10-01T19:56:53.701 INFO:teuthology.orchestra.run.smithi057.stdout: mon: 3 daemons, quorum a,c,b (age 16m) 2023-10-01T19:56:53.701 INFO:teuthology.orchestra.run.smithi057.stdout: mgr: x(active, since 6m), standbys: y 2023-10-01T19:56:53.701 INFO:teuthology.orchestra.run.smithi057.stdout: osd: 8 osds: 8 up (since 14m), 8 in (since 14m) 2023-10-01T19:56:53.701 INFO:teuthology.orchestra.run.smithi057.stdout: rgw: 2 daemons active (2 hosts, 1 zones) 2023-10-01T19:56:53.701 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:56:53.701 INFO:teuthology.orchestra.run.smithi057.stdout: data: 2023-10-01T19:56:53.701 INFO:teuthology.orchestra.run.smithi057.stdout: pools: 6 pools, 161 pgs 2023-10-01T19:56:53.701 INFO:teuthology.orchestra.run.smithi057.stdout: objects: 209 objects, 585 KiB 2023-10-01T19:56:53.701 INFO:teuthology.orchestra.run.smithi057.stdout: usage: 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:56:53.701 INFO:teuthology.orchestra.run.smithi057.stdout: pgs: 161 active+clean 2023-10-01T19:56:53.701 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:56:53.702 INFO:teuthology.orchestra.run.smithi057.stdout: io: 2023-10-01T19:56:53.702 INFO:teuthology.orchestra.run.smithi057.stdout: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2023-10-01T19:56:53.702 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T19:56:54.104 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph mgr fail' 2023-10-01T19:56:54.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:53 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3751894860' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T19:56:54.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:53 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3751894860' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T19:56:54.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:53 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3751894860' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T19:56:55.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:55 smithi057 ceph-mon[108286]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:55.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:55 smithi057 ceph-mon[114331]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:55.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:55 smithi087 ceph-mon[109386]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:56:56.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:56 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3905256469' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T19:56:56.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:56 smithi057 ceph-mon[108286]: osdmap e99: 8 total, 8 up, 8 in 2023-10-01T19:56:56.290 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:56:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:19:56:56] ENGINE Bus STOPPING 2023-10-01T19:56:56.290 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:56:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:19:56:56] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-10-01T19:56:56.290 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:56:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:19:56:56] ENGINE Bus STOPPED 2023-10-01T19:56:56.290 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:56:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:19:56:56] ENGINE Bus STARTING 2023-10-01T19:56:56.290 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:56:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:19:56:56] ENGINE Serving on http://:::9283 2023-10-01T19:56:56.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:56 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3905256469' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T19:56:56.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:56 smithi057 ceph-mon[114331]: osdmap e99: 8 total, 8 up, 8 in 2023-10-01T19:56:56.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:56 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3905256469' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T19:56:56.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:56 smithi087 ceph-mon[109386]: osdmap e99: 8 total, 8 up, 8 in 2023-10-01T19:56:56.354 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:56.013+0000 7f931fcdf700 -1 mgr handle_mgr_map I was active but no longer am 2023-10-01T19:56:56.354 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ignoring --setuser ceph since I am not root 2023-10-01T19:56:56.354 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ignoring --setgroup ceph since I am not root 2023-10-01T19:56:56.355 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:56 smithi087 ceph-mgr[137104]: -- 172.21.15.87:0/1874387928 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0d64ff1e0 con 0x55d0d64d5000 2023-10-01T19:56:56.355 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:56.189+0000 7fdef6109200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T19:56:56.355 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:56.353+0000 7fdef6109200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T19:56:56.439 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'sleep 180' 2023-10-01T19:56:56.589 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:56:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:19:56:56] ENGINE Bus STARTED 2023-10-01T19:56:56.882 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:56.687+0000 7fdef6109200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T19:56:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3905256469' entity='client.admin' cmd='[{"prefix": "mgr fail"}]': finished 2023-10-01T19:56:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: mgrmap e29: y(active, starting, since 0.303639s) 2023-10-01T19:56:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:56:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:56:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:56:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:56:57.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: Manager daemon y is now available 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:57.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:56:57.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:56:57.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:56:57.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:56:57.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:57.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:56:57.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:56:57.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:56:57.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:56:57.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:56:57.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:56:57.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:56:57.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:56:57.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[114331]: [01/Oct/2023:19:56:56] ENGINE Bus STARTING 2023-10-01T19:56:57.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3905256469' entity='client.admin' cmd='[{"prefix": "mgr fail"}]': finished 2023-10-01T19:56:57.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: mgrmap e29: y(active, starting, since 0.303639s) 2023-10-01T19:56:57.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:56:57.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:56:57.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:56:57.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:56:57.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:56:57.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:56:57.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:56:57.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:56:57.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:56:57.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:56:57.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:56:57.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:56:57.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:56:57.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:56:57.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:56:57.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: Manager daemon y is now available 2023-10-01T19:56:57.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:57.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:56:57.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:56:57.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:56:57.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:56:57.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:57.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:56:57.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:56:57.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:56:57.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:56:57.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:56:57.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:56:57.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:56:57.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:56:57.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:57 smithi057 ceph-mon[108286]: [01/Oct/2023:19:56:56] ENGINE Bus STARTING 2023-10-01T19:56:57.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:57.248+0000 7fdef6109200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T19:56:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3905256469' entity='client.admin' cmd='[{"prefix": "mgr fail"}]': finished 2023-10-01T19:56:57.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: mgrmap e29: y(active, starting, since 0.303639s) 2023-10-01T19:56:57.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T19:56:57.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T19:56:57.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T19:56:57.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T19:56:57.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T19:56:57.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T19:56:57.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T19:56:57.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T19:56:57.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T19:56:57.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T19:56:57.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T19:56:57.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T19:56:57.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T19:56:57.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T19:56:57.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T19:56:57.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: Manager daemon y is now available 2023-10-01T19:56:57.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:57.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:56:57.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:56:57.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T19:56:57.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T19:56:57.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:56:57.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T19:56:57.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T19:56:57.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:56:57.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T19:56:57.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:56:57.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:56:57.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T19:56:57.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T19:56:57.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-mon[109386]: [01/Oct/2023:19:56:56] ENGINE Bus STARTING 2023-10-01T19:56:57.797 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:57 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:57.504+0000 7fdef6109200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-01T19:56:58.063 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:58.021+0000 7fdef6109200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T19:56:58.063 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: [01/Oct/2023:19:56:56] ENGINE Serving on http://172.21.15.57:8765 2023-10-01T19:56:58.063 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: [01/Oct/2023:19:56:56] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:56:58.063 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: [01/Oct/2023:19:56:56] ENGINE Bus STARTED 2023-10-01T19:56:58.063 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T19:56:58.063 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T19:56:58.064 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: mgrmap e30: y(active, since 1.31568s) 2023-10-01T19:56:58.064 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:56:58.064 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:56:58.064 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.064 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:56:58.064 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.064 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T19:56:58.064 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.064 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.064 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.064 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.064 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.065 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.065 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: [01/Oct/2023:19:56:56] ENGINE Serving on http://172.21.15.57:8765 2023-10-01T19:56:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: [01/Oct/2023:19:56:56] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:56:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: [01/Oct/2023:19:56:56] ENGINE Bus STARTED 2023-10-01T19:56:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T19:56:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T19:56:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: mgrmap e30: y(active, since 1.31568s) 2023-10-01T19:56:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:56:58.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:56:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:56:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T19:56:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: [01/Oct/2023:19:56:56] ENGINE Serving on http://172.21.15.57:8765 2023-10-01T19:56:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: [01/Oct/2023:19:56:56] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T19:56:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: [01/Oct/2023:19:56:56] ENGINE Bus STARTED 2023-10-01T19:56:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T19:56:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T19:56:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: mgrmap e30: y(active, since 1.31568s) 2023-10-01T19:56:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:56:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:56:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T19:56:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T19:56:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:58 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:58.926 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:58.831+0000 7fdef6109200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T19:56:58.926 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:58.925+0000 7fdef6109200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T19:56:59.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T19:56:59.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:56:59.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.324 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.324 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:56:59.324 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.324 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: Reconfiguring iscsi.foo.smithi057.zfdcts (dependencies changed)... 2023-10-01T19:56:59.324 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T19:56:59.324 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:56:59.324 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: Reconfiguring daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T19:56:59.324 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.324 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.324 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T19:56:59.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:56:59.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:56:59.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: Reconfiguring iscsi.foo.smithi057.zfdcts (dependencies changed)... 2023-10-01T19:56:59.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T19:56:59.326 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:56:59.326 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: Reconfiguring daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T19:56:59.326 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.326 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.326 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:56:59 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.331 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T19:56:59.331 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:56:59.331 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.331 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T19:56:59.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: Reconfiguring iscsi.foo.smithi057.zfdcts (dependencies changed)... 2023-10-01T19:56:59.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T19:56:59.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:56:59.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: Reconfiguring daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T19:56:59.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:56:59.633 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:59.568+0000 7fdef6109200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T19:56:59.944 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:59.657+0000 7fdef6109200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T19:56:59.944 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:59.755+0000 7fdef6109200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T19:57:00.196 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:56:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:56:59.997+0000 7fdef6109200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T19:57:00.197 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:00.097+0000 7fdef6109200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T19:57:00.197 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-mon[109386]: mgrmap e31: y(active, since 3s) 2023-10-01T19:57:00.197 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.197 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.197 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.197 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.197 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.197 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.197 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.198 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[114331]: mgrmap e31: y(active, since 3s) 2023-10-01T19:57:00.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.278 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[108286]: mgrmap e31: y(active, since 3s) 2023-10-01T19:57:00.278 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.278 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.278 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.278 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.278 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.278 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.278 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.278 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:00 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:00.477 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:00.195+0000 7fdef6109200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T19:57:00.477 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:00.439+0000 7fdef6109200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T19:57:00.749 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:00.546+0000 7fdef6109200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T19:57:01.086 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:00.748+0000 7fdef6109200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T19:57:01.086 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:00.842+0000 7fdef6109200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T19:57:01.376 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:01 smithi057 ceph-mon[108286]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:57:01.376 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:01 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:01.377 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:01 smithi057 ceph-mon[114331]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:57:01.377 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:01 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:01.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:01 smithi087 ceph-mon[109386]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:57:01.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:01 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:01.792 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:01 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:01.641+0000 7fdef6109200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T19:57:02.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:01 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:01.849+0000 7fdef6109200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T19:57:02.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:02.033+0000 7fdef6109200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:57:02.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[114331]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:57:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[114331]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T19:57:02.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[108286]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:57:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:02 smithi057 ceph-mon[108286]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T19:57:03.097 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:03.097 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:03.097 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:03.097 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:02 smithi087 ceph-mon[109386]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T19:57:03.097 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:03.097 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:03.097 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:02 smithi087 ceph-mon[109386]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T19:57:03.098 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:02.697+0000 7fdef6109200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T19:57:03.098 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:02.897+0000 7fdef6109200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T19:57:03.098 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:03.096+0000 7fdef6109200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T19:57:03.377 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:03.225+0000 7fdef6109200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T19:57:03.377 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:57:03] ENGINE Bus STARTING 2023-10-01T19:57:03.377 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: CherryPy Checker: 2023-10-01T19:57:03.377 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: The Application mounted at '' has an empty config. 2023-10-01T19:57:03.377 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T19:57:03.378 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:57:03] ENGINE Serving on http://:::9283 2023-10-01T19:57:03.378 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:19:57:03] ENGINE Bus STARTED 2023-10-01T19:57:03.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:03.377Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.87:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.87:8765: connect: connection refused" 2023-10-01T19:57:03.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:03.377Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.87:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.87:8765: connect: connection refused" 2023-10-01T19:57:03.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:03.378Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.87:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.87:8765: connect: connection refused" 2023-10-01T19:57:03.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:03.378Z caller=refresh.go:98 level=error component="discovery manager notify" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.87:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.87:8765: connect: connection refused" 2023-10-01T19:57:04.092 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-mon[109386]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T19:57:04.093 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2227555337' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T19:57:04.093 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-mon[109386]: Standby manager daemon x started 2023-10-01T19:57:04.093 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T19:57:04.093 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:57:04.093 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T19:57:04.093 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:03 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:57:04.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[108286]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T19:57:04.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2227555337' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T19:57:04.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[108286]: Standby manager daemon x started 2023-10-01T19:57:04.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T19:57:04.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:57:04.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T19:57:04.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:57:04.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[114331]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T19:57:04.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2227555337' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T19:57:04.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[114331]: Standby manager daemon x started 2023-10-01T19:57:04.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T19:57:04.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T19:57:04.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T19:57:04.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:03 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T19:57:04.350 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 systemd[1]: Stopping Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:57:04.351 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 bash[151534]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus.a" found: no such container 2023-10-01T19:57:04.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:04.411Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2023-10-01T19:57:04.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:04.412Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2023-10-01T19:57:04.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:04.412Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2023-10-01T19:57:04.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:04.412Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2023-10-01T19:57:04.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:04.412Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2023-10-01T19:57:04.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:04.412Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2023-10-01T19:57:04.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:04.412Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2023-10-01T19:57:04.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:04.412Z caller=main.go:828 level=info msg="Scrape manager stopped" 2023-10-01T19:57:04.633 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:04.412Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2023-10-01T19:57:04.634 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:04.412Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2023-10-01T19:57:04.634 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:04.412Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2023-10-01T19:57:04.634 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[142590]: ts=2023-10-01T19:57:04.413Z caller=main.go:1066 level=info msg="See you next time!" 2023-10-01T19:57:04.634 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 bash[151548]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a 2023-10-01T19:57:04.634 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 bash[151585]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus.a" found: no such container 2023-10-01T19:57:05.133 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@prometheus.a.service: Succeeded. 2023-10-01T19:57:05.133 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 systemd[1]: Stopped Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:57:05.133 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:04 smithi087 systemd[1]: Starting Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T19:57:05.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-mon[109386]: mgrmap e32: y(active, since 8s), standbys: x 2023-10-01T19:57:05.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T19:57:05.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:05.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:05.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:05.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:04 smithi087 ceph-mon[109386]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T19:57:05.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:04 smithi057 ceph-mon[108286]: mgrmap e32: y(active, since 8s), standbys: x 2023-10-01T19:57:05.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:04 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T19:57:05.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:04 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:05.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:04 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:05.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:04 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:05.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:04 smithi057 ceph-mon[108286]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T19:57:05.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:04 smithi057 ceph-mon[114331]: mgrmap e32: y(active, since 8s), standbys: x 2023-10-01T19:57:05.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:04 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T19:57:05.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:04 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:05.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:04 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:05.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:04 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:05.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:04 smithi057 ceph-mon[114331]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T19:57:05.609 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.145Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2023-10-01T19:57:05.609 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.145Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2023-10-01T19:57:05.609 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 bash[151650]: 8dd0a211878635283b26084c5ce67b9c45d51380e52c451b934be8f88f46c74f 2023-10-01T19:57:05.609 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.145Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2023-10-01T19:57:05.609 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.145Z caller=main.go:518 level=info host_details="(Linux 4.18.0-514.el8.x86_64 #1 SMP Tue Sep 19 13:19:58 UTC 2023 x86_64 smithi087 (none))" 2023-10-01T19:57:05.609 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.145Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2023-10-01T19:57:05.610 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.145Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2023-10-01T19:57:05.610 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.147Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2023-10-01T19:57:05.610 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.147Z caller=main.go:923 level=info msg="Starting TSDB ..." 2023-10-01T19:57:05.610 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.148Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2023-10-01T19:57:05.610 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 systemd[1]: Started Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T19:57:05.610 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.151Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2023-10-01T19:57:05.610 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.151Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.88µs 2023-10-01T19:57:05.610 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.151Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2023-10-01T19:57:05.610 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.152Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=3 2023-10-01T19:57:05.610 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.609Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=3 2023-10-01T19:57:05.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.668Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=3 2023-10-01T19:57:05.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.669Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=3 2023-10-01T19:57:05.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.669Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=54.633µs wal_replay_duration=517.563107ms total_replay_duration=517.639833ms 2023-10-01T19:57:05.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.672Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2023-10-01T19:57:05.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.672Z caller=main.go:947 level=info msg="TSDB started" 2023-10-01T19:57:05.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.672Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2023-10-01T19:57:05.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.688Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.668337ms db_storage=690ns remote_storage=1.3µs web_handler=361ns query_engine=741ns scrape=428.689µs scrape_sd=75.811µs notify=17.03µs notify_sd=9.386µs rules=14.894945ms 2023-10-01T19:57:05.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 19:57:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T19:57:05.688Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2023-10-01T19:57:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:57:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:57:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: Adding iSCSI gateway http://:@172.21.15.57:5000 to Dashboard 2023-10-01T19:57:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:57:06.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:57:06.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:57:06.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:57:06.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:57:06.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:57:06.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:57:06.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:57:06.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:57:06.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:57:06.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:57:06.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: Adding iSCSI gateway http://:@172.21.15.57:5000 to Dashboard 2023-10-01T19:57:06.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:57:06.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:57:06.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:57:06.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:57:06.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:57:06.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:57:06.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:57:06.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:57:06.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:57:06.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:57:06.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T19:57:06.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: Adding iSCSI gateway http://:@172.21.15.57:5000 to Dashboard 2023-10-01T19:57:06.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:57:06.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T19:57:06.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:57:06.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T19:57:06.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:06.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:57:06.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T19:57:06.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:57:06.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:57:06.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:57:06.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:07.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:07 smithi087 ceph-mon[109386]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2023-10-01T19:57:07.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:07 smithi057 ceph-mon[108286]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2023-10-01T19:57:07.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:07 smithi057 ceph-mon[114331]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2023-10-01T19:57:09.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:09 smithi087 ceph-mon[109386]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T19:57:09.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:09 smithi057 ceph-mon[108286]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T19:57:09.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:09 smithi057 ceph-mon[114331]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T19:57:11.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:10 smithi087 ceph-mon[109386]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T19:57:11.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:10 smithi057 ceph-mon[108286]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T19:57:11.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:10 smithi057 ceph-mon[114331]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T19:57:12.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:11 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:57:12.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:11 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:57:12.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:11 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:57:13.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:12 smithi087 ceph-mon[109386]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T19:57:13.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:12 smithi057 ceph-mon[108286]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T19:57:13.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:12 smithi057 ceph-mon[114331]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T19:57:14.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:13 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:14.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:13 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:14.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:13 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:14.973 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:57:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:57:14] "GET /metrics HTTP/1.1" 200 37898 "" "Prometheus/2.33.4" 2023-10-01T19:57:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:14 smithi087 ceph-mon[109386]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T19:57:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:14 smithi057 ceph-mon[108286]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T19:57:15.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:14 smithi057 ceph-mon[114331]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T19:57:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:17 smithi087 ceph-mon[109386]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:17.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:17 smithi057 ceph-mon[108286]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:17.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:17 smithi057 ceph-mon[114331]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:19.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:19 smithi087 ceph-mon[109386]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:19.387 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:19 smithi057 ceph-mon[108286]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:19.387 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:19 smithi057 ceph-mon[114331]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:21.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:20 smithi087 ceph-mon[109386]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:21.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:20 smithi057 ceph-mon[108286]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:21.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:20 smithi057 ceph-mon[114331]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:22.366 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:22 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:22.366 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:22 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:22.366 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:22 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:22.366 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:22 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:22.366 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:22 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:22.367 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:22 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:22.548 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:22 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:22.548 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:22 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:22.548 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:22 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:23.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:23 smithi057 ceph-mon[108286]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:23.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:23 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:23.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:23 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:23.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:23 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:23.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:23 smithi057 ceph-mon[114331]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:23.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:23 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:23.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:23 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:23.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:23 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:23 smithi087 ceph-mon[109386]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:23 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:23 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:23 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:24.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:24 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:24.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:24 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:24.923 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:57:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:57:24] "GET /metrics HTTP/1.1" 200 37898 "" "Prometheus/2.33.4" 2023-10-01T19:57:24.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:24 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:25.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:25 smithi087 ceph-mon[109386]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:25.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:25 smithi057 ceph-mon[108286]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:25.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:25 smithi057 ceph-mon[114331]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:26.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:26 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:57:26.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:26 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:57:26.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:26 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:57:27.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:27 smithi087 ceph-mon[109386]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:27.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:27 smithi057 ceph-mon[108286]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:27.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:27 smithi057 ceph-mon[114331]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:29.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:29 smithi087 ceph-mon[109386]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:29.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:29 smithi057 ceph-mon[108286]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:29.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:29 smithi057 ceph-mon[114331]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:31.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:30 smithi087 ceph-mon[109386]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:31.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:30 smithi057 ceph-mon[108286]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:31.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:30 smithi057 ceph-mon[114331]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:33 smithi087 ceph-mon[109386]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:33.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:33 smithi057 ceph-mon[108286]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:33 smithi057 ceph-mon[114331]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:34.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:34 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:34.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:34 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:34.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:34 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:35.105 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:57:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:57:34] "GET /metrics HTTP/1.1" 200 37893 "" "Prometheus/2.33.4" 2023-10-01T19:57:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:35 smithi087 ceph-mon[109386]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:35.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:35 smithi057 ceph-mon[108286]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:35 smithi057 ceph-mon[114331]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:37.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:37 smithi087 ceph-mon[109386]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:37.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:37 smithi057 ceph-mon[108286]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:37.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:37 smithi057 ceph-mon[114331]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:39 smithi087 ceph-mon[109386]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:39.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:39 smithi057 ceph-mon[108286]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:39.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:39 smithi057 ceph-mon[114331]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:41.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:41 smithi057 ceph-mon[108286]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:41.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:41 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:41.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:41 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:41.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:41 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:41.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:41 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:57:41.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:41 smithi057 ceph-mon[114331]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:41.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:41 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:41.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:41 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:41.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:41 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:41.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:41 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:57:41.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:41 smithi087 ceph-mon[109386]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:41.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:41 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:41.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:41 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:41.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:41 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:41.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:41 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:57:43.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:42 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:43.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:42 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:43.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:42 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:43.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:42 smithi057 ceph-mon[108286]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:43.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:42 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:43.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:42 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:43.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:42 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:43.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:42 smithi057 ceph-mon[114331]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:43.132 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:42 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:43.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:42 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:43.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:42 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:57:43.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:42 smithi087 ceph-mon[109386]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:44.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:43 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:44.120 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:43 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:44.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:43 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:45.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:44 smithi087 ceph-mon[109386]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:45.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:44 smithi057 ceph-mon[108286]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:45.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:57:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:57:44] "GET /metrics HTTP/1.1" 200 37891 "" "Prometheus/2.33.4" 2023-10-01T19:57:45.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:44 smithi057 ceph-mon[114331]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:47.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:47 smithi087 ceph-mon[109386]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:47.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:47 smithi057 ceph-mon[108286]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:47.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:47 smithi057 ceph-mon[114331]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:49 smithi087 ceph-mon[109386]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:49.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:49 smithi057 ceph-mon[108286]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:49.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:49 smithi057 ceph-mon[114331]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:51.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:50 smithi087 ceph-mon[109386]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:51.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:50 smithi057 ceph-mon[108286]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:51.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:50 smithi057 ceph-mon[114331]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:53.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:53 smithi087 ceph-mon[109386]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:53.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:53 smithi057 ceph-mon[108286]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:53.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:53 smithi057 ceph-mon[114331]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:54.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:54 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:54.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:54 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:54.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:54 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:57:55.117 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:57:54 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:57:54] "GET /metrics HTTP/1.1" 200 37891 "" "Prometheus/2.33.4" 2023-10-01T19:57:55.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:55 smithi087 ceph-mon[109386]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:55.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:55 smithi057 ceph-mon[108286]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:55.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:55 smithi057 ceph-mon[114331]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:57 smithi087 ceph-mon[109386]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:57:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:57 smithi057 ceph-mon[108286]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:57:57.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:57 smithi057 ceph-mon[114331]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:57:57.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:57:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:57:59 smithi087 ceph-mon[109386]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:59.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:57:59 smithi057 ceph-mon[108286]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:57:59.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:57:59 smithi057 ceph-mon[114331]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:01.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:00 smithi087 ceph-mon[109386]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:01.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:00 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:01.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:00 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:01.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:00 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:01.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:00 smithi057 ceph-mon[108286]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:00 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:00 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:00 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:01.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:00 smithi057 ceph-mon[114331]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:01.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:00 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:01.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:00 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:01.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:00 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:03.111 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:03.111 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:03.111 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:03.112 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:02 smithi087 ceph-mon[109386]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:03.132 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:03.133 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:03.133 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:03.133 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:02 smithi057 ceph-mon[108286]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:03.133 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:03.133 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:03.133 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:03.133 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:02 smithi057 ceph-mon[114331]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:04.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:03 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:04.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:03 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:04.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:03 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:05.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:04 smithi087 ceph-mon[109386]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:05.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:04 smithi057 ceph-mon[108286]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:05.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:58:04 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:58:04] "GET /metrics HTTP/1.1" 200 37892 "" "Prometheus/2.33.4" 2023-10-01T19:58:05.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:04 smithi057 ceph-mon[114331]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:06.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:05 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:58:06.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:05 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:58:06.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:05 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:58:06.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:05 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:06.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:05 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:58:06.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:05 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:58:06.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:05 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:58:06.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:05 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:06.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:05 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:58:06.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:05 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:58:06.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:05 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:58:06.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:05 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:07.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:06 smithi087 ceph-mon[109386]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:07.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:06 smithi057 ceph-mon[108286]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:06 smithi057 ceph-mon[114331]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:09 smithi087 ceph-mon[109386]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:09.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:09 smithi057 ceph-mon[108286]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:09.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:09 smithi057 ceph-mon[114331]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:11.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:10 smithi087 ceph-mon[109386]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:11.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:10 smithi057 ceph-mon[108286]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:11.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:10 smithi057 ceph-mon[114331]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:12.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:11 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:58:12.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:11 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:58:12.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:11 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:58:13.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:12 smithi057 ceph-mon[108286]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:13.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:12 smithi057 ceph-mon[114331]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:13.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:12 smithi087 ceph-mon[109386]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:14.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:13 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:14.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:13 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:14.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:13 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:14.962 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:58:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:58:14] "GET /metrics HTTP/1.1" 200 37892 "" "Prometheus/2.33.4" 2023-10-01T19:58:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:14 smithi087 ceph-mon[109386]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:14 smithi057 ceph-mon[108286]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:15.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:14 smithi057 ceph-mon[114331]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:17.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:17 smithi087 ceph-mon[109386]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:17.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:17 smithi057 ceph-mon[108286]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:17.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:17 smithi057 ceph-mon[114331]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:19.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:19 smithi087 ceph-mon[109386]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:19.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:19 smithi057 ceph-mon[108286]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:19.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:19 smithi057 ceph-mon[114331]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:20 smithi087 ceph-mon[109386]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:20 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:20 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:20 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:20 smithi057 ceph-mon[108286]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:20 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:20 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:20 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:20 smithi057 ceph-mon[114331]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:21.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:20 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:21.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:20 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:21.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:20 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:23.254 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:23 smithi087 ceph-mon[109386]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:23.255 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:23 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:23.255 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:23 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:23.255 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:23 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:23.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:23 smithi057 ceph-mon[108286]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:23.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:23 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:23.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:23 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:23.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:23 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:23.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:23 smithi057 ceph-mon[114331]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:23.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:23 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:23.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:23 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:23.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:23 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:24.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:24 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:24.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:24 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:24.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:24 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:24.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:58:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:58:24] "GET /metrics HTTP/1.1" 200 37892 "" "Prometheus/2.33.4" 2023-10-01T19:58:25.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:25 smithi087 ceph-mon[109386]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:25.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:25 smithi057 ceph-mon[108286]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:25.472 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:25 smithi057 ceph-mon[114331]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:27.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:27 smithi087 ceph-mon[109386]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:27.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:27 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:58:27.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:27 smithi057 ceph-mon[114331]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:27.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:27 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:58:27.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:27 smithi057 ceph-mon[108286]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:27.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:27 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:58:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:29 smithi087 ceph-mon[109386]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:29.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:29 smithi057 ceph-mon[108286]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:29.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:29 smithi057 ceph-mon[114331]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:31.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:30 smithi087 ceph-mon[109386]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:31.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:30 smithi057 ceph-mon[108286]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:31.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:30 smithi057 ceph-mon[114331]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:33 smithi087 ceph-mon[109386]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:33.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:33 smithi057 ceph-mon[108286]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:33 smithi057 ceph-mon[114331]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:34.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:34 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:34.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:34 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:34 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:35.116 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:58:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:58:34] "GET /metrics HTTP/1.1" 200 37892 "" "Prometheus/2.33.4" 2023-10-01T19:58:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:35 smithi087 ceph-mon[109386]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:35.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:35 smithi057 ceph-mon[108286]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:35 smithi057 ceph-mon[114331]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:37.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:37 smithi087 ceph-mon[109386]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:37.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:37 smithi057 ceph-mon[108286]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:37.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:37 smithi057 ceph-mon[114331]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:39.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:39 smithi057 ceph-mon[108286]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:39.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:39 smithi057 ceph-mon[114331]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:39.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:39 smithi087 ceph-mon[109386]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:41.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:40 smithi087 ceph-mon[109386]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:41.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:40 smithi057 ceph-mon[108286]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:41.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:40 smithi057 ceph-mon[114331]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:42.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:41 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:42.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:41 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:42.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:41 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:42.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:41 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:58:42.256 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:41 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:42.256 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:41 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:42.256 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:41 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:42.256 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:41 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:58:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:41 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:41 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:41 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:41 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:58:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:43 smithi057 ceph-mon[108286]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:43 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:43 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:43 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:43 smithi057 ceph-mon[114331]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:43 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:43 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:43 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:43.510 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:43 smithi087 ceph-mon[109386]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:43.510 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:43 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:43.510 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:43 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:43.510 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:43 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:58:44.579 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:44 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:44.579 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:44 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:44.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:44 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:44.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:58:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:58:44] "GET /metrics HTTP/1.1" 200 37891 "" "Prometheus/2.33.4" 2023-10-01T19:58:45.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:45 smithi087 ceph-mon[109386]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:45.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:45 smithi057 ceph-mon[108286]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:45.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:45 smithi057 ceph-mon[114331]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:47.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:47 smithi087 ceph-mon[109386]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:47.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:47 smithi057 ceph-mon[108286]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:47.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:47 smithi057 ceph-mon[114331]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:49.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:49 smithi087 ceph-mon[109386]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:49.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:49 smithi057 ceph-mon[108286]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:49.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:49 smithi057 ceph-mon[114331]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:51.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:50 smithi087 ceph-mon[109386]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:51.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:50 smithi057 ceph-mon[108286]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:51.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:50 smithi057 ceph-mon[114331]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:53.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:53 smithi087 ceph-mon[109386]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:53.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:53 smithi057 ceph-mon[108286]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:53.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:53 smithi057 ceph-mon[114331]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:54.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:54 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:54.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:54 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:54.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:54 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:58:55.123 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:58:54 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:58:54] "GET /metrics HTTP/1.1" 200 37891 "" "Prometheus/2.33.4" 2023-10-01T19:58:55.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:55 smithi087 ceph-mon[109386]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:55.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:55 smithi057 ceph-mon[108286]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:55.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:55 smithi057 ceph-mon[114331]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:57.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:57 smithi087 ceph-mon[109386]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:57 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:58:57.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:57 smithi057 ceph-mon[114331]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:57.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:57 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:58:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:57 smithi057 ceph-mon[108286]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:58:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:57 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:58:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:58:59 smithi087 ceph-mon[109386]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:59.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:58:59 smithi057 ceph-mon[108286]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:58:59.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:58:59 smithi057 ceph-mon[114331]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:01.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:00 smithi087 ceph-mon[109386]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:01.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:00 smithi057 ceph-mon[108286]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:01.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:00 smithi057 ceph-mon[114331]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:02.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:02.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:02.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:02.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:02.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:02.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:02.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:02.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:02.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:03 smithi057 ceph-mon[108286]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:03 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:03 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:03 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:03 smithi057 ceph-mon[114331]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:03 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:03 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:03 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:03 smithi087 ceph-mon[109386]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:03 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:03 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:03 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:04.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:04 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:04.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:04 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:04.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:04 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:05.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:59:04 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:59:04] "GET /metrics HTTP/1.1" 200 37892 "" "Prometheus/2.33.4" 2023-10-01T19:59:05.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:05 smithi057 ceph-mon[108286]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:05.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:05 smithi057 ceph-mon[114331]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:05 smithi087 ceph-mon[109386]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:06.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:59:06.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:59:06.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:59:06.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:06.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:59:06.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:59:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:59:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T19:59:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T19:59:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T19:59:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:07.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:07 smithi087 ceph-mon[109386]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:07.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:07 smithi057 ceph-mon[108286]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:07.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:07 smithi057 ceph-mon[114331]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:09.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:09 smithi087 ceph-mon[109386]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:09.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:09 smithi057 ceph-mon[108286]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:09.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:09 smithi057 ceph-mon[114331]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:11.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:10 smithi087 ceph-mon[109386]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:11.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:10 smithi057 ceph-mon[108286]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:11.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:10 smithi057 ceph-mon[114331]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:12.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:11 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:59:12.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:11 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:59:12.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:11 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:59:13.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:12 smithi057 ceph-mon[108286]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:13.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:12 smithi057 ceph-mon[114331]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:12 smithi087 ceph-mon[109386]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:14.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:13 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:14.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:13 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:14.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:13 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:14.972 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:59:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:59:14] "GET /metrics HTTP/1.1" 200 37900 "" "Prometheus/2.33.4" 2023-10-01T19:59:15.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:14 smithi087 ceph-mon[109386]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:14 smithi057 ceph-mon[108286]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:15.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:14 smithi057 ceph-mon[114331]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:17.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:17 smithi057 ceph-mon[108286]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:17.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:17 smithi057 ceph-mon[114331]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:17.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:17 smithi087 ceph-mon[109386]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:19.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:19 smithi057 ceph-mon[108286]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:19.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:19 smithi057 ceph-mon[114331]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:19.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:19 smithi087 ceph-mon[109386]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:21.272 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:20 smithi087 ceph-mon[109386]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:21.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:20 smithi057 ceph-mon[108286]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:20 smithi057 ceph-mon[114331]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:22.598 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:22 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:22.598 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:22 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:22.598 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:22 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:22.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:22 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:22.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:22 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:22.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:22 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:22 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:22 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:22 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:23 smithi087 ceph-mon[109386]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:23 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:23 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:23 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:23.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:23 smithi057 ceph-mon[108286]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:23.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:23 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:23.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:23 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:23.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:23 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:23 smithi057 ceph-mon[114331]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:23 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:23 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:23 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:24.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:24 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:24.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:24 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:24.923 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:59:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:59:24] "GET /metrics HTTP/1.1" 200 37900 "" "Prometheus/2.33.4" 2023-10-01T19:59:24.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:24 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:25.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:25 smithi087 ceph-mon[109386]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:25.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:25 smithi057 ceph-mon[108286]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:25.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:25 smithi057 ceph-mon[114331]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:26 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:59:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:26 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:59:26.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:26 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:59:27.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:27 smithi087 ceph-mon[109386]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:27 smithi057 ceph-mon[108286]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:27.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:27 smithi057 ceph-mon[114331]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:28.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:28 smithi057 ceph-mon[108286]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:28.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:28 smithi057 ceph-mon[114331]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:29.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:28 smithi087 ceph-mon[109386]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:31.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:30 smithi087 ceph-mon[109386]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:31.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:30 smithi057 ceph-mon[108286]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:31.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:30 smithi057 ceph-mon[114331]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:33.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:33 smithi087 ceph-mon[109386]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:33.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:33 smithi057 ceph-mon[108286]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:33 smithi057 ceph-mon[114331]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:34.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:34 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:34.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:34 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:34.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:34 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:35.137 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:59:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:59:34] "GET /metrics HTTP/1.1" 200 37897 "" "Prometheus/2.33.4" 2023-10-01T19:59:35.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:35 smithi057 ceph-mon[108286]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:35.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:35 smithi057 ceph-mon[114331]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:35.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:35 smithi087 ceph-mon[109386]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:37.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:37 smithi057 ceph-mon[108286]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:37.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:37 smithi057 ceph-mon[114331]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:37.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:37 smithi087 ceph-mon[109386]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:39.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:39 smithi057 ceph-mon[108286]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:39.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:39 smithi057 ceph-mon[114331]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:39.449 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:39 smithi087 ceph-mon[109386]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:41.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:40 smithi057 ceph-mon[108286]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:41.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:40 smithi057 ceph-mon[114331]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:41.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:40 smithi087 ceph-mon[109386]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:41 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:59:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:41 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:41 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:41 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:42.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:41 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:59:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:41 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:41 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:41 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:42.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:41 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:59:42.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:41 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:42.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:41 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:42.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:41 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:43.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:42 smithi057 ceph-mon[108286]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:43.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:42 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:43.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:42 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:43.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:42 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:43.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:42 smithi057 ceph-mon[114331]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:43.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:42 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:43.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:42 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:43.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:42 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:43.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:42 smithi087 ceph-mon[109386]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:42 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:42 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:42 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T19:59:44.321 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:43 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:44.322 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:43 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:44.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:43 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:44.943 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:59:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:59:44] "GET /metrics HTTP/1.1" 200 37899 "" "Prometheus/2.33.4" 2023-10-01T19:59:45.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:44 smithi087 ceph-mon[109386]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:45.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:44 smithi057 ceph-mon[108286]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:44 smithi057 ceph-mon[114331]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:47.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:47 smithi087 ceph-mon[109386]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:47.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:47 smithi057 ceph-mon[108286]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:47.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:47 smithi057 ceph-mon[114331]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:49.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:49 smithi057 ceph-mon[108286]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:49.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:49 smithi057 ceph-mon[114331]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:49.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:49 smithi087 ceph-mon[109386]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:51.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:50 smithi087 ceph-mon[109386]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:51.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:50 smithi057 ceph-mon[108286]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:51.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:50 smithi057 ceph-mon[114331]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:53.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:53 smithi087 ceph-mon[109386]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:53.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:53 smithi057 ceph-mon[108286]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:53.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:53 smithi057 ceph-mon[114331]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:54.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:54 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:54.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:54 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:54 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T19:59:55.139 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 19:59:54 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:19:59:54] "GET /metrics HTTP/1.1" 200 37899 "" "Prometheus/2.33.4" 2023-10-01T19:59:55.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:55 smithi057 ceph-mon[108286]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:55.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:55 smithi057 ceph-mon[114331]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:55.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:55 smithi087 ceph-mon[109386]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:56.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:56 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:59:56.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:56 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:59:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:56 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T19:59:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:57 smithi057 ceph-mon[108286]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:57.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:57 smithi057 ceph-mon[114331]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:57.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:57 smithi087 ceph-mon[109386]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T19:59:58.484 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch ps' 2023-10-01T19:59:59.577 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 19:59:59 smithi057 ceph-mon[108286]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:59.578 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 19:59:59 smithi057 ceph-mon[114331]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T19:59:59.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 19:59:59 smithi087 ceph-mon[109386]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:00.008 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T20:00:00.008 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 17s ago 20m - - 2023-10-01T20:00:00.008 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 18s ago 20m - - 2023-10-01T20:00:00.008 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (9m) 17s ago 16m 27.6M - 0.23.0 ba2b418f427c 0b16afb678d9 2023-10-01T20:00:00.008 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (8m) 18s ago 16m 58.1M - 8.3.5 dad864ee21e9 ca8ad8b1ccf8 2023-10-01T20:00:00.008 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (2m) 17s ago 15m 58.5M - 3.5 e1d6a67b021e 7dcd89e78b4a 2023-10-01T20:00:00.008 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283 running (11m) 18s ago 19m 453M - 18.0.0-6505-g37d71fd3 bea6d6056140 f7b2c168ac86 2023-10-01T20:00:00.008 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (6m) 17s ago 21m 514M - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T20:00:00.009 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (21m) 17s ago 21m 159M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T20:00:00.009 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (20m) 18s ago 20m 97.3M 2048M 17.2.0 e1d6a67b021e 4734e639b775 2023-10-01T20:00:00.009 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (20m) 17s ago 20m 85.3M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T20:00:00.009 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (9m) 17s ago 16m 24.9M - 1.3.1 1dbe0e931976 41af659015b7 2023-10-01T20:00:00.009 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (8m) 18s ago 16m 27.1M - 1.3.1 1dbe0e931976 dd38d7fd60ea 2023-10-01T20:00:00.009 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (19m) 17s ago 19m 143M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T20:00:00.009 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (19m) 17s ago 19m 150M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T20:00:00.009 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (18m) 17s ago 18m 111M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T20:00:00.009 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (18m) 17s ago 18m 162M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T20:00:00.009 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (18m) 18s ago 18m 156M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T20:00:00.009 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (18m) 18s ago 18m 150M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T20:00:00.009 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (17m) 18s ago 17m 122M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T20:00:00.010 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (17m) 18s ago 17m 148M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T20:00:00.010 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (2m) 18s ago 17m 62.0M - 2.33.4 514e6a882f6e 8dd0a2118786 2023-10-01T20:00:00.010 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (16m) 17s ago 16m 98.1M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T20:00:00.010 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (16m) 18s ago 16m 94.3M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T20:00:00.386 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions' 2023-10-01T20:00:01.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:00 smithi057 ceph-mon[114331]: from='client.25129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:00:01.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:00 smithi057 ceph-mon[114331]: overall HEALTH_OK 2023-10-01T20:00:01.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:00 smithi057 ceph-mon[114331]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:01.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:00 smithi057 ceph-mon[108286]: from='client.25129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:00:01.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:00 smithi057 ceph-mon[108286]: overall HEALTH_OK 2023-10-01T20:00:01.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:00 smithi057 ceph-mon[108286]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:01.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:00 smithi087 ceph-mon[109386]: from='client.25129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:00:01.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:00 smithi087 ceph-mon[109386]: overall HEALTH_OK 2023-10-01T20:00:01.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:00 smithi087 ceph-mon[109386]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:02.012 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:00:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2023-10-01T20:00:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2023-10-01T20:00:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:00:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2023-10-01T20:00:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:00:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:00:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2023-10-01T20:00:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2023-10-01T20:00:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:00:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": {}, 2023-10-01T20:00:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "rgw": { 2023-10-01T20:00:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T20:00:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:00:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2023-10-01T20:00:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 13, 2023-10-01T20:00:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:00:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T20:00:02.014 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:00:02.508 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph -s' 2023-10-01T20:00:02.837 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:02.837 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:02.837 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:02 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:02.837 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:02 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1759406575' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:00:02.838 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:02.838 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:02.838 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:02 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:02.839 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:02 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1759406575' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:00:02.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:02.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:02.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:02 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:02.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:02 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1759406575' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:00:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:03 smithi087 ceph-mon[109386]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:03 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:03 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:03 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:03.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:03 smithi057 ceph-mon[108286]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:03.891 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:03 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:03.891 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:03 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:03.891 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:03 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:03.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:03 smithi057 ceph-mon[114331]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:03.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:03 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:03.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:03 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:03.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:03 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:04.418 INFO:teuthology.orchestra.run.smithi057.stdout: cluster: 2023-10-01T20:00:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: id: 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T20:00:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: health: HEALTH_OK 2023-10-01T20:00:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T20:00:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: services: 2023-10-01T20:00:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: mon: 3 daemons, quorum a,c,b (age 20m) 2023-10-01T20:00:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: mgr: y(active, since 3m), standbys: x 2023-10-01T20:00:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: osd: 8 osds: 8 up (since 17m), 8 in (since 17m) 2023-10-01T20:00:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: rgw: 2 daemons active (2 hosts, 1 zones) 2023-10-01T20:00:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T20:00:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: data: 2023-10-01T20:00:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: pools: 6 pools, 161 pgs 2023-10-01T20:00:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: objects: 209 objects, 585 KiB 2023-10-01T20:00:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: usage: 90 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: pgs: 161 active+clean 2023-10-01T20:00:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T20:00:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: io: 2023-10-01T20:00:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2023-10-01T20:00:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-01T20:00:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:04 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:04 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2027040138' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T20:00:04.903 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:04 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:04.903 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:04 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2027040138' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T20:00:04.903 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:04 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:00:04] "GET /metrics HTTP/1.1" 200 37901 "" "Prometheus/2.33.4" 2023-10-01T20:00:04.904 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:04 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:04.904 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:04 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2027040138' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-01T20:00:05.084 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph health detail' 2023-10-01T20:00:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:05 smithi087 ceph-mon[109386]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:05.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:05 smithi057 ceph-mon[108286]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:05.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:05 smithi057 ceph-mon[114331]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:06.904 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:00:06.904 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:06.904 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:06.904 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:00:06.904 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:00:06.904 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[108286]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:06.905 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[108286]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:06.905 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:00:06.905 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:06.905 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:06.905 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:00:06.905 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:00:06.905 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[114331]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:06.905 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:06 smithi057 ceph-mon[114331]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:06.923 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2023-10-01T20:00:07.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:00:07.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:07.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:07.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:00:07.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:00:07.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:06 smithi087 ceph-mon[109386]: from='mgr.15141 172.21.15.57:0/3310031641' entity='mgr.y' 2023-10-01T20:00:07.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:06 smithi087 ceph-mon[109386]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:07.276 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2023-10-01T20:00:07.646 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:07 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/67344289' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T20:00:07.646 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:07 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/67344289' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T20:00:07.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:07 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/67344289' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T20:00:08.661 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:08 smithi057 ceph-mon[114331]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:08.874 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:00:08.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:08 smithi057 ceph-mon[108286]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:09.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:08 smithi087 ceph-mon[109386]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:09.230 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph mgr fail' 2023-10-01T20:00:09.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:09 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1485236354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:00:09.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:09 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1485236354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:00:10.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:09 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1485236354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:00:11.329 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ignoring --setuser ceph since I am not root 2023-10-01T20:00:11.329 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ignoring --setgroup ceph since I am not root 2023-10-01T20:00:11.329 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mgr[161953]: -- 172.21.15.57:0/2649619236 <== mon.1 v2:172.21.15.57:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ca545e31e0 con 0x55ca545b8c00 2023-10-01T20:00:11.330 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:11.191+0000 7f4d07f46200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T20:00:11.330 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:10 smithi057 ceph-mon[108286]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:11.330 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:10 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3790248586' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T20:00:11.330 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:10 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T20:00:11.330 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:10 smithi057 ceph-mon[108286]: osdmap e100: 8 total, 8 up, 8 in 2023-10-01T20:00:11.330 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:10 smithi057 ceph-mon[114331]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:11.331 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:10 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3790248586' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T20:00:11.331 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:10 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T20:00:11.331 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:10 smithi057 ceph-mon[114331]: osdmap e100: 8 total, 8 up, 8 in 2023-10-01T20:00:11.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:10 smithi087 ceph-mon[109386]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 90 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:11.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:10 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3790248586' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T20:00:11.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:10 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr fail"}]: dispatch 2023-10-01T20:00:11.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:10 smithi087 ceph-mon[109386]: osdmap e100: 8 total, 8 up, 8 in 2023-10-01T20:00:11.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:00:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:20:00:10] ENGINE Bus STOPPING 2023-10-01T20:00:11.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:20:00:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-10-01T20:00:11.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:20:00:11] ENGINE Bus STOPPED 2023-10-01T20:00:11.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:20:00:11] ENGINE Bus STARTING 2023-10-01T20:00:11.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:20:00:11] ENGINE Serving on http://:::9283 2023-10-01T20:00:11.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:20:00:11] ENGINE Bus STARTED 2023-10-01T20:00:11.401 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'sleep 180' 2023-10-01T20:00:11.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:11.399+0000 7f4d07f46200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T20:00:12.038 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr fail"}]': finished 2023-10-01T20:00:12.038 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: mgrmap e33: x(active, starting, since 0.182623s) 2023-10-01T20:00:12.038 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:00:12.038 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:00:12.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:00:12.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T20:00:12.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:00:12.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T20:00:12.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T20:00:12.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T20:00:12.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T20:00:12.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T20:00:12.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T20:00:12.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T20:00:12.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T20:00:12.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T20:00:12.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T20:00:12.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: Manager daemon x is now available 2023-10-01T20:00:12.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:12.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:00:12.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:00:12.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T20:00:12.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T20:00:12.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:00:12.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-01T20:00:12.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-01T20:00:12.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:00:12.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:12.041 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:00:12.041 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:00:12.041 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:00:12.122 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:11.791+0000 7f4d07f46200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T20:00:12.123 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr fail"}]': finished 2023-10-01T20:00:12.123 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: mgrmap e33: x(active, starting, since 0.182623s) 2023-10-01T20:00:12.123 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:00:12.123 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:00:12.123 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:00:12.124 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T20:00:12.124 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:00:12.124 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T20:00:12.124 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T20:00:12.124 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T20:00:12.124 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T20:00:12.124 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T20:00:12.124 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T20:00:12.124 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T20:00:12.124 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T20:00:12.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T20:00:12.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T20:00:12.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: Manager daemon x is now available 2023-10-01T20:00:12.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:12.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:00:12.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:00:12.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T20:00:12.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T20:00:12.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:00:12.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-01T20:00:12.126 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-01T20:00:12.126 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:00:12.126 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:12.126 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:00:12.126 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:00:12.126 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:00:12.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr fail"}]': finished 2023-10-01T20:00:12.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: mgrmap e33: x(active, starting, since 0.182623s) 2023-10-01T20:00:12.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:00:12.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:00:12.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:00:12.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T20:00:12.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:00:12.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T20:00:12.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T20:00:12.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T20:00:12.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T20:00:12.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T20:00:12.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T20:00:12.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T20:00:12.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T20:00:12.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T20:00:12.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T20:00:12.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: Manager daemon x is now available 2023-10-01T20:00:12.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:12.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:00:12.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:00:12.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T20:00:12.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T20:00:12.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:00:12.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-01T20:00:12.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-01T20:00:12.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:00:12.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:12.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:00:12.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:00:12.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:00:12.749 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:12.449+0000 7f4d07f46200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T20:00:13.004 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:12.748+0000 7f4d07f46200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-01T20:00:13.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: [01/Oct/2023:20:00:11] ENGINE Bus STARTING 2023-10-01T20:00:13.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: [01/Oct/2023:20:00:11] ENGINE Serving on https://172.21.15.87:7150 2023-10-01T20:00:13.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: [01/Oct/2023:20:00:11] ENGINE Serving on http://172.21.15.87:8765 2023-10-01T20:00:13.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: [01/Oct/2023:20:00:11] ENGINE Bus STARTED 2023-10-01T20:00:13.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T20:00:13.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T20:00:13.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: mgrmap e34: x(active, since 1.2044s) 2023-10-01T20:00:13.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:13.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: [01/Oct/2023:20:00:11] ENGINE Bus STARTING 2023-10-01T20:00:13.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: [01/Oct/2023:20:00:11] ENGINE Serving on https://172.21.15.87:7150 2023-10-01T20:00:13.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: [01/Oct/2023:20:00:11] ENGINE Serving on http://172.21.15.87:8765 2023-10-01T20:00:13.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: [01/Oct/2023:20:00:11] ENGINE Bus STARTED 2023-10-01T20:00:13.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T20:00:13.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T20:00:13.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: mgrmap e34: x(active, since 1.2044s) 2023-10-01T20:00:13.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:13.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: [01/Oct/2023:20:00:11] ENGINE Bus STARTING 2023-10-01T20:00:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: [01/Oct/2023:20:00:11] ENGINE Serving on https://172.21.15.87:7150 2023-10-01T20:00:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: [01/Oct/2023:20:00:11] ENGINE Serving on http://172.21.15.87:8765 2023-10-01T20:00:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: [01/Oct/2023:20:00:11] ENGINE Bus STARTED 2023-10-01T20:00:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T20:00:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T20:00:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: mgrmap e34: x(active, since 1.2044s) 2023-10-01T20:00:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:13.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:13 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:13.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:13.348+0000 7f4d07f46200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T20:00:14.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:00:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:00:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:00:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:00:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:00:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:00:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: mgrmap e35: x(active, since 2s) 2023-10-01T20:00:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T20:00:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:00:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.328 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:00:14.328 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:00:14.328 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:00:14.329 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:00:14.329 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:14.329 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:00:14.329 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:14.329 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:00:14.329 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.329 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.329 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: mgrmap e35: x(active, since 2s) 2023-10-01T20:00:14.329 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.329 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.329 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.329 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.330 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.330 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T20:00:14.330 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:00:14.330 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.330 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.330 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:14 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.368 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:14.265+0000 7f4d07f46200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T20:00:14.368 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:00:14.368 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:00:14.368 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:00:14.368 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:00:14.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='client.25133 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:14.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:00:14.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:14.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:00:14.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: mgrmap e35: x(active, since 2s) 2023-10-01T20:00:14.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T20:00:14.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:00:14.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:14.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:14.367+0000 7f4d07f46200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T20:00:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:15 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:00:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:15 smithi087 ceph-mon[109386]: Reconfiguring iscsi.foo.smithi057.zfdcts (dependencies changed)... 2023-10-01T20:00:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:15 smithi087 ceph-mon[109386]: Reconfiguring daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T20:00:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:15 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:15 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:15.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:15 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:15.321 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:15.116+0000 7f4d07f46200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T20:00:15.321 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:15.227+0000 7f4d07f46200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T20:00:15.322 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:00:15.322 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-mon[108286]: Reconfiguring iscsi.foo.smithi057.zfdcts (dependencies changed)... 2023-10-01T20:00:15.322 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-mon[108286]: Reconfiguring daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T20:00:15.322 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:15.322 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:15.322 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:15.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:00:15.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-mon[114331]: Reconfiguring iscsi.foo.smithi057.zfdcts (dependencies changed)... 2023-10-01T20:00:15.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-mon[114331]: Reconfiguring daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T20:00:15.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:15.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:15.323 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:15.607 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:15.337+0000 7f4d07f46200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T20:00:15.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:15.607+0000 7f4d07f46200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T20:00:15.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:15.717+0000 7f4d07f46200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T20:00:15.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:15 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:15.840+0000 7f4d07f46200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T20:00:16.363 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:16.101+0000 7f4d07f46200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T20:00:16.363 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:16.224+0000 7f4d07f46200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T20:00:16.363 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-mon[108286]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:16.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:16.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:16.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:16.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-mon[108286]: mgrmap e36: x(active, since 4s) 2023-10-01T20:00:16.364 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-mon[114331]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:16.364 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:16.364 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:16.364 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:16.364 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-mon[114331]: mgrmap e36: x(active, since 4s) 2023-10-01T20:00:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:16 smithi087 ceph-mon[109386]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:16 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:16 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:16 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:16 smithi087 ceph-mon[109386]: mgrmap e36: x(active, since 4s) 2023-10-01T20:00:16.621 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:16.446+0000 7f4d07f46200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T20:00:16.622 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:16.553+0000 7f4d07f46200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T20:00:17.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:17 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:17.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:17 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:17.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:17 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:17.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:17 smithi057 ceph-mon[108286]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:17.259 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:17 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:17.259 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:17 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:17.259 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:17 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:17.259 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:17 smithi057 ceph-mon[114331]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:17.604 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:17 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:17.503+0000 7f4d07f46200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T20:00:17.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:17 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:17.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:17 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:17.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:17 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:17.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:17 smithi087 ceph-mon[109386]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:00:17.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:17 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:17.760+0000 7f4d07f46200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T20:00:18.376 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:17 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:17.980+0000 7f4d07f46200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T20:00:18.628 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:18.628 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:18.628 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-mon[114331]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T20:00:18.628 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-mon[114331]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T20:00:18.628 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/538011392' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T20:00:18.629 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2419688091' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2119020240"}]: dispatch 2023-10-01T20:00:18.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:18.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:18.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-mon[108286]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T20:00:18.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-mon[108286]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T20:00:18.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/538011392' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T20:00:18.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2419688091' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2119020240"}]: dispatch 2023-10-01T20:00:18.632 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:18 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:18 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:18 smithi087 ceph-mon[109386]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T20:00:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:18 smithi087 ceph-mon[109386]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T20:00:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:18 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/538011392' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T20:00:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:18 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2419688091' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2119020240"}]: dispatch 2023-10-01T20:00:18.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:18 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:18.807+0000 7f4d07f46200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T20:00:19.251 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:19.030+0000 7f4d07f46200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T20:00:19.260 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 systemd[1]: Stopping Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:00:19.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2419688091' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2119020240"}]': finished 2023-10-01T20:00:19.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[108286]: osdmap e101: 8 total, 8 up, 8 in 2023-10-01T20:00:19.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1636503472' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3878196940"}]: dispatch 2023-10-01T20:00:19.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:19.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:19.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:19.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[108286]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T20:00:19.508 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:19.250+0000 7f4d07f46200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T20:00:19.508 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:19.387+0000 7f4d07f46200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T20:00:19.508 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:00:19] ENGINE Bus STARTING 2023-10-01T20:00:19.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2419688091' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2119020240"}]': finished 2023-10-01T20:00:19.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[114331]: osdmap e101: 8 total, 8 up, 8 in 2023-10-01T20:00:19.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1636503472' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3878196940"}]: dispatch 2023-10-01T20:00:19.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:19.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:19.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:19.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-mon[114331]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T20:00:19.553 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2419688091' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6801/2119020240"}]': finished 2023-10-01T20:00:19.553 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-mon[109386]: osdmap e101: 8 total, 8 up, 8 in 2023-10-01T20:00:19.554 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1636503472' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3878196940"}]: dispatch 2023-10-01T20:00:19.554 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:19.554 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:19.554 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:19.554 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-mon[109386]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T20:00:19.554 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 bash[156747]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus.a" found: no such container 2023-10-01T20:00:19.554 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T20:00:19.310Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2023-10-01T20:00:19.555 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T20:00:19.311Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2023-10-01T20:00:19.555 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T20:00:19.311Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2023-10-01T20:00:19.555 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T20:00:19.311Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2023-10-01T20:00:19.555 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T20:00:19.311Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2023-10-01T20:00:19.555 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T20:00:19.311Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2023-10-01T20:00:19.555 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T20:00:19.311Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2023-10-01T20:00:19.555 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T20:00:19.311Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2023-10-01T20:00:19.555 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T20:00:19.311Z caller=main.go:828 level=info msg="Scrape manager stopped" 2023-10-01T20:00:19.555 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T20:00:19.312Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2023-10-01T20:00:19.555 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T20:00:19.312Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2023-10-01T20:00:19.555 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[151664]: ts=2023-10-01T20:00:19.312Z caller=main.go:1066 level=info msg="See you next time!" 2023-10-01T20:00:19.556 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 bash[156759]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a 2023-10-01T20:00:19.811 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 bash[156797]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus.a" found: no such container 2023-10-01T20:00:19.811 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@prometheus.a.service: Succeeded. 2023-10-01T20:00:19.812 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 systemd[1]: Stopped Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:00:19.812 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 systemd[1]: Starting Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:00:19.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: CherryPy Checker: 2023-10-01T20:00:19.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: The Application mounted at '' has an empty config. 2023-10-01T20:00:19.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:00:19.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:00:19] ENGINE Serving on http://:::9283 2023-10-01T20:00:19.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:00:19 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:00:19] ENGINE Bus STARTED 2023-10-01T20:00:20.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.935Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2023-10-01T20:00:20.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.935Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2023-10-01T20:00:20.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.935Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2023-10-01T20:00:20.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.935Z caller=main.go:518 level=info host_details="(Linux 4.18.0-514.el8.x86_64 #1 SMP Tue Sep 19 13:19:58 UTC 2023 x86_64 smithi087 (none))" 2023-10-01T20:00:20.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.935Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2023-10-01T20:00:20.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.935Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2023-10-01T20:00:20.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.937Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2023-10-01T20:00:20.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.937Z caller=main.go:923 level=info msg="Starting TSDB ..." 2023-10-01T20:00:20.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.938Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2023-10-01T20:00:20.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 bash[156865]: 844c1a351b1e295a503952683c2ea01af27e6b408d54059940e0d9f00c73cc37 2023-10-01T20:00:20.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 systemd[1]: Started Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:00:20.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.942Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2023-10-01T20:00:20.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.942Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=747.862µs 2023-10-01T20:00:20.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.942Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2023-10-01T20:00:20.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:19.944Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=4 2023-10-01T20:00:20.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-01T20:00:20.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: Standby manager daemon y started 2023-10-01T20:00:20.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T20:00:20.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1636503472' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3878196940"}]': finished 2023-10-01T20:00:20.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: osdmap e102: 8 total, 8 up, 8 in 2023-10-01T20:00:20.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-01T20:00:20.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T20:00:20.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3037336909' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2119020240"}]: dispatch 2023-10-01T20:00:20.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2119020240"}]: dispatch 2023-10-01T20:00:20.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:20.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:20.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T20:00:20.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T20:00:20.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T20:00:20.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T20:00:20.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:00:20.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:00:20.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:00:20.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:00:20.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:00:20.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:20.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-01T20:00:20.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: Standby manager daemon y started 2023-10-01T20:00:20.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T20:00:20.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1636503472' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3878196940"}]': finished 2023-10-01T20:00:20.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: osdmap e102: 8 total, 8 up, 8 in 2023-10-01T20:00:20.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-01T20:00:20.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T20:00:20.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3037336909' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2119020240"}]: dispatch 2023-10-01T20:00:20.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2119020240"}]: dispatch 2023-10-01T20:00:20.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:20.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:20.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T20:00:20.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T20:00:20.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T20:00:20.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T20:00:20.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:00:20.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:00:20.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:00:20.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:00:20.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:00:20.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:20.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:20.421Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=4 2023-10-01T20:00:20.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:20.480Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=4 2023-10-01T20:00:20.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:20.533Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=4 2023-10-01T20:00:20.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:20.533Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=4 maxSegment=4 2023-10-01T20:00:20.885 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:20.533Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=45.924µs wal_replay_duration=590.768762ms total_replay_duration=591.590052ms 2023-10-01T20:00:20.885 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:20.537Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2023-10-01T20:00:20.885 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:20.537Z caller=main.go:947 level=info msg="TSDB started" 2023-10-01T20:00:20.885 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:20.537Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2023-10-01T20:00:20.885 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:20.556Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=19.396337ms db_storage=955ns remote_storage=1.423µs web_handler=361ns query_engine=676ns scrape=433.91µs scrape_sd=72.767µs notify=15.227µs notify_sd=14.661µs rules=18.626336ms 2023-10-01T20:00:20.886 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:00:20.557Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2023-10-01T20:00:20.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-01T20:00:20.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: Standby manager daemon y started 2023-10-01T20:00:20.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T20:00:20.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1636503472' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3878196940"}]': finished 2023-10-01T20:00:20.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: osdmap e102: 8 total, 8 up, 8 in 2023-10-01T20:00:20.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-01T20:00:20.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T20:00:20.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3037336909' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2119020240"}]: dispatch 2023-10-01T20:00:20.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2119020240"}]: dispatch 2023-10-01T20:00:20.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:20.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:20.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T20:00:20.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T20:00:20.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T20:00:20.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T20:00:20.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:00:20.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:00:20.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:00:20.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:00:20.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:00:20.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:21.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:21 smithi057 ceph-mon[108286]: mgrmap e37: x(active, since 9s), standbys: y 2023-10-01T20:00:21.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:21 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T20:00:21.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:21 smithi057 ceph-mon[108286]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2119020240"}]': finished 2023-10-01T20:00:21.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:21 smithi057 ceph-mon[108286]: osdmap e103: 8 total, 8 up, 8 in 2023-10-01T20:00:21.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:21 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2564649405' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3838981579"}]: dispatch 2023-10-01T20:00:21.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:21 smithi057 ceph-mon[108286]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 20 op/s 2023-10-01T20:00:21.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:21 smithi057 ceph-mon[114331]: mgrmap e37: x(active, since 9s), standbys: y 2023-10-01T20:00:21.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:21 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T20:00:21.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:21 smithi057 ceph-mon[114331]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2119020240"}]': finished 2023-10-01T20:00:21.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:21 smithi057 ceph-mon[114331]: osdmap e103: 8 total, 8 up, 8 in 2023-10-01T20:00:21.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:21 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2564649405' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3838981579"}]: dispatch 2023-10-01T20:00:21.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:21 smithi057 ceph-mon[114331]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 20 op/s 2023-10-01T20:00:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:21 smithi087 ceph-mon[109386]: mgrmap e37: x(active, since 9s), standbys: y 2023-10-01T20:00:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:21 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T20:00:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:21 smithi087 ceph-mon[109386]: from='client.? ' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:6800/2119020240"}]': finished 2023-10-01T20:00:21.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:21 smithi087 ceph-mon[109386]: osdmap e103: 8 total, 8 up, 8 in 2023-10-01T20:00:21.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:21 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2564649405' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3838981579"}]: dispatch 2023-10-01T20:00:21.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:21 smithi087 ceph-mon[109386]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 20 op/s 2023-10-01T20:00:22.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:22 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2564649405' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3838981579"}]': finished 2023-10-01T20:00:22.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:22 smithi087 ceph-mon[109386]: osdmap e104: 8 total, 8 up, 8 in 2023-10-01T20:00:22.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:22 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1114556338' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2683065235"}]: dispatch 2023-10-01T20:00:22.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:22 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2564649405' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3838981579"}]': finished 2023-10-01T20:00:22.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:22 smithi057 ceph-mon[108286]: osdmap e104: 8 total, 8 up, 8 in 2023-10-01T20:00:22.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:22 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1114556338' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2683065235"}]: dispatch 2023-10-01T20:00:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:22 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2564649405' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/3838981579"}]': finished 2023-10-01T20:00:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:22 smithi057 ceph-mon[114331]: osdmap e104: 8 total, 8 up, 8 in 2023-10-01T20:00:22.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:22 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1114556338' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2683065235"}]: dispatch 2023-10-01T20:00:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:23 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1114556338' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2683065235"}]': finished 2023-10-01T20:00:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:23 smithi087 ceph-mon[109386]: osdmap e105: 8 total, 8 up, 8 in 2023-10-01T20:00:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:23 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2501540086' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1588655190"}]: dispatch 2023-10-01T20:00:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:23 smithi087 ceph-mon[109386]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:23.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:23 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1114556338' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2683065235"}]': finished 2023-10-01T20:00:23.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:23 smithi057 ceph-mon[108286]: osdmap e105: 8 total, 8 up, 8 in 2023-10-01T20:00:23.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:23 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2501540086' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1588655190"}]: dispatch 2023-10-01T20:00:23.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:23 smithi057 ceph-mon[108286]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:23 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1114556338' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/2683065235"}]': finished 2023-10-01T20:00:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:23 smithi057 ceph-mon[114331]: osdmap e105: 8 total, 8 up, 8 in 2023-10-01T20:00:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:23 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2501540086' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1588655190"}]: dispatch 2023-10-01T20:00:23.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:23 smithi057 ceph-mon[114331]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:24.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:24 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2501540086' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1588655190"}]': finished 2023-10-01T20:00:24.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:24 smithi087 ceph-mon[109386]: osdmap e106: 8 total, 8 up, 8 in 2023-10-01T20:00:24.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:24 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2501540086' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1588655190"}]': finished 2023-10-01T20:00:24.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:24 smithi057 ceph-mon[108286]: osdmap e106: 8 total, 8 up, 8 in 2023-10-01T20:00:24.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:24 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2501540086' entity='client.iscsi.foo.smithi057.zfdcts' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.57:0/1588655190"}]': finished 2023-10-01T20:00:24.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:24 smithi057 ceph-mon[114331]: osdmap e106: 8 total, 8 up, 8 in 2023-10-01T20:00:26.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:25 smithi087 ceph-mon[109386]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:00:26.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:25 smithi057 ceph-mon[108286]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:00:26.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:25 smithi057 ceph-mon[114331]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:00:27.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:26 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:00:27.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:26 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:00:27.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:26 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:00:28.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:27 smithi087 ceph-mon[109386]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:28.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:27 smithi057 ceph-mon[108286]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:28.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:27 smithi057 ceph-mon[114331]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:29 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:29.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:29 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:29.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:29 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:30 smithi087 ceph-mon[109386]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2023-10-01T20:00:30.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:30 smithi057 ceph-mon[108286]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2023-10-01T20:00:30.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:30 smithi057 ceph-mon[114331]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2023-10-01T20:00:31.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:00:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:00:30] "GET /metrics HTTP/1.1" 200 37895 "" "Prometheus/2.33.4" 2023-10-01T20:00:32.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:32 smithi087 ceph-mon[109386]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 598 B/s rd, 0 op/s 2023-10-01T20:00:32.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:32 smithi057 ceph-mon[108286]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 598 B/s rd, 0 op/s 2023-10-01T20:00:32.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:32 smithi057 ceph-mon[114331]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 598 B/s rd, 0 op/s 2023-10-01T20:00:34.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:34 smithi087 ceph-mon[109386]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-10-01T20:00:34.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:34 smithi057 ceph-mon[108286]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-10-01T20:00:34.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:34 smithi057 ceph-mon[114331]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-10-01T20:00:36.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:35 smithi087 ceph-mon[109386]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 887 B/s rd, 0 op/s 2023-10-01T20:00:36.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:35 smithi057 ceph-mon[108286]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 887 B/s rd, 0 op/s 2023-10-01T20:00:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:35 smithi057 ceph-mon[114331]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 887 B/s rd, 0 op/s 2023-10-01T20:00:37.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.630 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.630 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[108286]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:37.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.630 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:37 smithi057 ceph-mon[114331]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:37.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:37 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:37 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:37 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:37 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:37 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:37 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:37.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:37 smithi087 ceph-mon[109386]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:38.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:38 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:38 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:38 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:38.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:38 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:38.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:38 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:38.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:38 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:38.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:38 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:38.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:38 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:38.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:38 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:38.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:38 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:38.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:38 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:38.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:38 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:39.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:39 smithi057 ceph-mon[108286]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:39.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:39 smithi057 ceph-mon[114331]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:39.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:39 smithi087 ceph-mon[109386]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:41.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:00:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:00:40] "GET /metrics HTTP/1.1" 200 37895 "" "Prometheus/2.33.4" 2023-10-01T20:00:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:42 smithi087 ceph-mon[109386]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:42 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:00:42.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:42 smithi057 ceph-mon[108286]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:42.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:42 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:00:42.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:42 smithi057 ceph-mon[114331]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:42.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:42 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:00:44.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:44 smithi087 ceph-mon[109386]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:44.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:44 smithi057 ceph-mon[114331]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:44 smithi057 ceph-mon[108286]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:45 smithi087 ceph-mon[109386]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:46.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:45 smithi057 ceph-mon[108286]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:46.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:45 smithi057 ceph-mon[114331]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:48 smithi087 ceph-mon[109386]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:48.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:48 smithi057 ceph-mon[108286]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:48.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:48 smithi057 ceph-mon[114331]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:49 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:49.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:49 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:49.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:49 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:50.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:50 smithi087 ceph-mon[109386]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:50.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:50 smithi057 ceph-mon[108286]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:50 smithi057 ceph-mon[114331]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:51.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:00:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:00:50] "GET /metrics HTTP/1.1" 200 37902 "" "Prometheus/2.33.4" 2023-10-01T20:00:52.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:52 smithi087 ceph-mon[109386]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:52.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:52 smithi057 ceph-mon[108286]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:52.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:52 smithi057 ceph-mon[114331]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:54.345 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:54 smithi087 ceph-mon[109386]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:54.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:54 smithi057 ceph-mon[108286]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:54.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:54 smithi057 ceph-mon[114331]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:00:56.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:55 smithi087 ceph-mon[109386]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:56.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:55 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:56.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:55 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:56.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:55 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:56.388 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:55 smithi057 ceph-mon[108286]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:56.388 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:55 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:56.388 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:55 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:56.388 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:55 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:56.388 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:55 smithi057 ceph-mon[114331]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:56.388 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:55 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:56.388 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:55 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:56.388 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:55 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:57.376 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:56 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:00:57.376 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:56 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:57.376 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:56 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:57.376 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:56 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:57.376 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:57 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:00:57.377 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:57 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:57.377 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:57 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:57.377 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:57 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:56 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:00:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:56 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:56 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:56 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:00:58.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:58 smithi087 ceph-mon[109386]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:58.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:58 smithi057 ceph-mon[108286]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:58.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:58 smithi057 ceph-mon[114331]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:00:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:00:59 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:59.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:00:59 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:00:59.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:00:59 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:00.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:00 smithi087 ceph-mon[109386]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:00.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:00 smithi057 ceph-mon[108286]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:00 smithi057 ceph-mon[114331]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:01.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:01:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:01:00] "GET /metrics HTTP/1.1" 200 37897 "" "Prometheus/2.33.4" 2023-10-01T20:01:02.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:02 smithi087 ceph-mon[109386]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:02.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:02 smithi057 ceph-mon[108286]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:02.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:02 smithi057 ceph-mon[114331]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:04.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:04 smithi087 ceph-mon[109386]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:04.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:04 smithi057 ceph-mon[108286]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:04.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:04 smithi057 ceph-mon[114331]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:06.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:05 smithi087 ceph-mon[109386]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:06.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:05 smithi057 ceph-mon[108286]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:06.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:05 smithi057 ceph-mon[114331]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:08.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:08 smithi087 ceph-mon[109386]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:08.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:08 smithi057 ceph-mon[108286]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:08.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:08 smithi057 ceph-mon[114331]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:09 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:09.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:09 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:09.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:09 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:10.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:10 smithi087 ceph-mon[109386]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:10.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:10 smithi057 ceph-mon[108286]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:10.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:10 smithi057 ceph-mon[114331]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:11.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:01:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:01:10] "GET /metrics HTTP/1.1" 200 37897 "" "Prometheus/2.33.4" 2023-10-01T20:01:12.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:12 smithi087 ceph-mon[109386]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:12.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:12 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:01:12.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:12 smithi057 ceph-mon[108286]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:12.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:12 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:01:12.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:12 smithi057 ceph-mon[114331]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:12.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:12 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:01:14.337 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:14 smithi087 ceph-mon[109386]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:14.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:14 smithi057 ceph-mon[114331]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:14.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:14 smithi057 ceph-mon[108286]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:16.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:15 smithi087 ceph-mon[109386]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:16.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:15 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:16.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:15 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:16.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:15 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:16.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:15 smithi057 ceph-mon[108286]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:16.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:15 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:16.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:15 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:16.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:15 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:15 smithi057 ceph-mon[114331]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:15 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:15 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:15 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:17 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:17 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:17 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:17 smithi087 ceph-mon[109386]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:18.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:17 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:18.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:17 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:18.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:17 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:18.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:17 smithi057 ceph-mon[108286]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:18.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:17 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:18.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:17 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:18.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:17 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:18.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:17 smithi057 ceph-mon[114331]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:19.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:18 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:19.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:18 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:19.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:18 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:20.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:19 smithi087 ceph-mon[109386]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:20.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:19 smithi057 ceph-mon[108286]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:20.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:19 smithi057 ceph-mon[114331]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:21.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:01:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:01:20] "GET /metrics HTTP/1.1" 200 37898 "" "Prometheus/2.33.4" 2023-10-01T20:01:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:01:21.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:21.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:21.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:01:21.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:01:21.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:01:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:01:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:01:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:01:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:21.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:21.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:01:21.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:01:21.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:21 smithi087 ceph-mon[109386]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:22.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:21 smithi057 ceph-mon[108286]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:22.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:21 smithi057 ceph-mon[114331]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:24.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:24 smithi087 ceph-mon[109386]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:24.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:24 smithi057 ceph-mon[108286]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:24.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:24 smithi057 ceph-mon[114331]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:26.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:25 smithi087 ceph-mon[109386]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:26.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:25 smithi057 ceph-mon[108286]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:26.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:25 smithi057 ceph-mon[114331]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:27.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:26 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:01:27.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:26 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:01:27.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:26 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:01:28.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:28 smithi087 ceph-mon[109386]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:28.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:28 smithi057 ceph-mon[108286]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:28.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:28 smithi057 ceph-mon[114331]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:29 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:29.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:29 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:29.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:29 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:30.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:30 smithi087 ceph-mon[109386]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:30.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:30 smithi057 ceph-mon[108286]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:30.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:30 smithi057 ceph-mon[114331]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:31.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:01:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:01:30] "GET /metrics HTTP/1.1" 200 37893 "" "Prometheus/2.33.4" 2023-10-01T20:01:32.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:32 smithi087 ceph-mon[109386]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:32.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:32 smithi057 ceph-mon[108286]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:32.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:32 smithi057 ceph-mon[114331]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:34.307 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:34 smithi087 ceph-mon[109386]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:34.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:34 smithi057 ceph-mon[108286]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:34.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:34 smithi057 ceph-mon[114331]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:36.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:35 smithi087 ceph-mon[109386]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:36.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:35 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:36.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:35 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:36.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:35 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:36.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:35 smithi057 ceph-mon[108286]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:36.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:35 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:36.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:35 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:36.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:35 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:36.359 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:35 smithi057 ceph-mon[114331]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:36.359 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:35 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:36.359 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:35 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:36.359 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:35 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:38.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:38 smithi087 ceph-mon[109386]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:38 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:38 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:38 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:38 smithi057 ceph-mon[108286]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:38 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:38 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:38 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:38.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:38 smithi057 ceph-mon[114331]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:38.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:38 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:38.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:38 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:38.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:38 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:39 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:39.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:39 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:39.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:39 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:40.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:40 smithi087 ceph-mon[109386]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:40.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:40 smithi057 ceph-mon[114331]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:40.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:40 smithi057 ceph-mon[108286]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:41.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:01:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:01:40] "GET /metrics HTTP/1.1" 200 37893 "" "Prometheus/2.33.4" 2023-10-01T20:01:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:42 smithi087 ceph-mon[109386]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:42 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:01:42.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:42 smithi057 ceph-mon[108286]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:42 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:01:42.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:42 smithi057 ceph-mon[114331]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:42.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:42 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:01:44.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:44 smithi087 ceph-mon[109386]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:44.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:44 smithi057 ceph-mon[108286]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:44.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:44 smithi057 ceph-mon[114331]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:45 smithi087 ceph-mon[109386]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:46.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:45 smithi057 ceph-mon[108286]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:46.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:45 smithi057 ceph-mon[114331]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:48 smithi087 ceph-mon[109386]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:48.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:48 smithi057 ceph-mon[108286]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:48.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:48 smithi057 ceph-mon[114331]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:49.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:49 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:49.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:49 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:49.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:49 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:50.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:50 smithi087 ceph-mon[109386]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:50.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:50 smithi057 ceph-mon[108286]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:50 smithi057 ceph-mon[114331]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:51.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:01:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:01:50] "GET /metrics HTTP/1.1" 200 37898 "" "Prometheus/2.33.4" 2023-10-01T20:01:51.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:51 smithi057 ceph-mon[108286]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:51.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:51 smithi057 ceph-mon[114331]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:51.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:51 smithi087 ceph-mon[109386]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:54.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:54 smithi087 ceph-mon[109386]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:54.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:54 smithi057 ceph-mon[108286]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:54.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:54 smithi057 ceph-mon[114331]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:01:56.348 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:55 smithi057 ceph-mon[108286]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:56.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:55 smithi057 ceph-mon[114331]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:56.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:55 smithi087 ceph-mon[109386]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:56 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:01:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:56 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:56 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:56 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:56 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:01:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:56 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:56 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:56 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:57.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:56 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:01:57.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:56 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:57.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:56 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:57.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:56 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:58.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:57 smithi057 ceph-mon[108286]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:58.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:57 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:58.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:57 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:58.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:57 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:58.343 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:57 smithi057 ceph-mon[114331]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:58.343 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:57 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:58.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:57 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:58.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:57 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:57 smithi087 ceph-mon[109386]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:01:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:57 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:57 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:57 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:01:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:01:58 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:59.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:01:58 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:01:59.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:01:58 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:00.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:00 smithi087 ceph-mon[109386]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:00.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:00 smithi057 ceph-mon[108286]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:00 smithi057 ceph-mon[114331]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:01.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:02:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:02:00] "GET /metrics HTTP/1.1" 200 37894 "" "Prometheus/2.33.4" 2023-10-01T20:02:02.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:02 smithi087 ceph-mon[109386]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:02.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:02 smithi057 ceph-mon[108286]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:02.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:02 smithi057 ceph-mon[114331]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:04.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:04 smithi087 ceph-mon[109386]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:04.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:04 smithi057 ceph-mon[108286]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:04.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:04 smithi057 ceph-mon[114331]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:06.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:06 smithi087 ceph-mon[109386]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:06.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:05 smithi057 ceph-mon[108286]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:06.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:06 smithi057 ceph-mon[114331]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:08.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:08 smithi087 ceph-mon[109386]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:08.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:08 smithi057 ceph-mon[108286]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:08.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:08 smithi057 ceph-mon[114331]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:09 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:09.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:09 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:09.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:09 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:10.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:10 smithi087 ceph-mon[109386]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:10.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:10 smithi057 ceph-mon[108286]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:10.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:10 smithi057 ceph-mon[114331]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:11.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:02:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:02:10] "GET /metrics HTTP/1.1" 200 37894 "" "Prometheus/2.33.4" 2023-10-01T20:02:11.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:11 smithi057 ceph-mon[108286]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:11.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:02:11.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:11 smithi057 ceph-mon[114331]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:11.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:02:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:11 smithi087 ceph-mon[109386]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:11.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:02:14.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:14 smithi087 ceph-mon[109386]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:14.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:14 smithi057 ceph-mon[108286]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:14.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:14 smithi057 ceph-mon[114331]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:16.273 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:15 smithi087 ceph-mon[109386]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:16.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:15 smithi057 ceph-mon[108286]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:15 smithi057 ceph-mon[114331]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:17.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:17 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:17.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:17 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:17.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:17 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:17.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:17 smithi057 ceph-mon[108286]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:17.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:17 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:17.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:17 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:17.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:17 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:17.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:17 smithi057 ceph-mon[114331]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:17.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:17 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:17.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:17 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:17.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:17 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:17.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:17 smithi087 ceph-mon[109386]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:18.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:18 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:18.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:18 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:18.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:18 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:18.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:18 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:18.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:18 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:18.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:18 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:18.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:18 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:18.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:18 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:18.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:18 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:18.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:18 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:18.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:18 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:18.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:18 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:19.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:19 smithi087 ceph-mon[109386]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:19.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:19 smithi057 ceph-mon[108286]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:19.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:19 smithi057 ceph-mon[114331]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:21.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:02:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:02:20] "GET /metrics HTTP/1.1" 200 37896 "" "Prometheus/2.33.4" 2023-10-01T20:02:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:02:21.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:02:21.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:02:21.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:20 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:21.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:02:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:02:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:02:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:20 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:02:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:02:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:02:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:20 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:22 smithi087 ceph-mon[109386]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:22.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:22 smithi057 ceph-mon[108286]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:22.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:22 smithi057 ceph-mon[114331]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:24.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:24 smithi087 ceph-mon[109386]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:24.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:24 smithi057 ceph-mon[108286]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:24.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:24 smithi057 ceph-mon[114331]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:26.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:26 smithi087 ceph-mon[109386]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:26.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:26 smithi057 ceph-mon[108286]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:26.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:26 smithi057 ceph-mon[114331]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:27.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:27 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:02:27.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:27 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:02:27.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:27 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:02:28.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:28 smithi087 ceph-mon[109386]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:28.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:28 smithi057 ceph-mon[108286]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:28.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:28 smithi057 ceph-mon[114331]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:29 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:29.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:29 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:29.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:29 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:30.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:30 smithi087 ceph-mon[109386]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:30.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:30 smithi057 ceph-mon[108286]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:30.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:30 smithi057 ceph-mon[114331]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:31.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:02:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:02:30] "GET /metrics HTTP/1.1" 200 37902 "" "Prometheus/2.33.4" 2023-10-01T20:02:32.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:32 smithi087 ceph-mon[109386]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:32.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:32 smithi057 ceph-mon[108286]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:32.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:32 smithi057 ceph-mon[114331]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:34.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:34 smithi087 ceph-mon[109386]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:34.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:34 smithi057 ceph-mon[114331]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:34 smithi057 ceph-mon[108286]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:36.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:36 smithi087 ceph-mon[109386]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:36.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:36 smithi057 ceph-mon[114331]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:36.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:36 smithi057 ceph-mon[108286]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:37.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:37 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:37.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:37 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:37.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:37 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:37.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:37 smithi057 ceph-mon[108286]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:37.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:37 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:37.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:37 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:37.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:37 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:37.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:37 smithi057 ceph-mon[114331]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:37.796 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:37 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:37.797 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:37 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:37.797 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:37 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:37.797 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:37 smithi087 ceph-mon[109386]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:38.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:38 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:38.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:38 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:38.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:38 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:38.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:38 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:38.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:38 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:38.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:38 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:38.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:38 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:38.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:38 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:38.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:38 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:38.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:38 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:38.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:38 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:38.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:38 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:39.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:39 smithi087 ceph-mon[109386]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:39.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:39 smithi057 ceph-mon[108286]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:39.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:39 smithi057 ceph-mon[114331]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:41.132 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:02:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:02:40] "GET /metrics HTTP/1.1" 200 37902 "" "Prometheus/2.33.4" 2023-10-01T20:02:41.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:41 smithi087 ceph-mon[109386]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:41.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:41 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:02:41.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:41 smithi057 ceph-mon[108286]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:41.456 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:41 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:02:41.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:41 smithi057 ceph-mon[114331]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:41.456 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:41 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:02:44.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:44 smithi087 ceph-mon[109386]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:44.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:44 smithi057 ceph-mon[108286]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:44.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:44 smithi057 ceph-mon[114331]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:46 smithi087 ceph-mon[109386]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:46.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:46 smithi057 ceph-mon[108286]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:46.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:46 smithi057 ceph-mon[114331]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:48 smithi087 ceph-mon[109386]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:48.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:48 smithi057 ceph-mon[108286]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:48.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:48 smithi057 ceph-mon[114331]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:49 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:49.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:49 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:49.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:49 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:50.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:50 smithi087 ceph-mon[109386]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:50.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:50 smithi057 ceph-mon[108286]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:50 smithi057 ceph-mon[114331]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:51.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:02:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:02:50] "GET /metrics HTTP/1.1" 200 37907 "" "Prometheus/2.33.4" 2023-10-01T20:02:52.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:52 smithi087 ceph-mon[109386]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:52.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:52 smithi057 ceph-mon[108286]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:52.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:52 smithi057 ceph-mon[114331]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:53.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:53 smithi057 ceph-mon[108286]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:53.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:53 smithi057 ceph-mon[114331]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:53.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:53 smithi087 ceph-mon[109386]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:56.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:56 smithi057 ceph-mon[108286]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:56.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:56 smithi057 ceph-mon[114331]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:56.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:56 smithi087 ceph-mon[109386]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:02:57.352 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:57 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:02:57.352 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:57 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:57.352 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:57 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:57.352 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:57 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:57.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:57 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:02:57.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:57 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:57.353 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:57 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:57.353 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:57 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:57 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:02:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:57 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:57 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:57.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:57 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:58 smithi087 ceph-mon[109386]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:58 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:58 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:58 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:58.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:58 smithi057 ceph-mon[108286]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:58.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:58 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:58.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:58 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:58.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:58 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:58.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:58 smithi057 ceph-mon[114331]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:02:58.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:58 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:58.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:58 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:58.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:58 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:02:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:02:59 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:59.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:02:59 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:02:59.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:02:59 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:00.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:00 smithi087 ceph-mon[109386]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:00.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:00 smithi057 ceph-mon[108286]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:00 smithi057 ceph-mon[114331]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:01.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:03:00] "GET /metrics HTTP/1.1" 200 37901 "" "Prometheus/2.33.4" 2023-10-01T20:03:02.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:02 smithi087 ceph-mon[109386]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:02.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:02 smithi057 ceph-mon[108286]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:02.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:02 smithi057 ceph-mon[114331]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:04.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:04 smithi087 ceph-mon[109386]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:04.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:04 smithi057 ceph-mon[108286]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:04.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:04 smithi057 ceph-mon[114331]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:06.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:06 smithi087 ceph-mon[109386]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:06.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:06 smithi057 ceph-mon[108286]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:06.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:06 smithi057 ceph-mon[114331]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:08.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:08 smithi087 ceph-mon[109386]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:08.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:08 smithi057 ceph-mon[108286]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:08.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:08 smithi057 ceph-mon[114331]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:09 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:09.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:09 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:09.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:09 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:10.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:10 smithi087 ceph-mon[109386]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:10.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:10 smithi057 ceph-mon[108286]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:10.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:10 smithi057 ceph-mon[114331]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:11.124 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:03:10] "GET /metrics HTTP/1.1" 200 37901 "" "Prometheus/2.33.4" 2023-10-01T20:03:11.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:11 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:11.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:11 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:11.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:11 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:12.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:12 smithi087 ceph-mon[109386]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:12.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:12 smithi057 ceph-mon[108286]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:12.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:12 smithi057 ceph-mon[114331]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:13.287 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch ps' 2023-10-01T20:03:13.321 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:13 smithi057 ceph-mon[108286]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:13.322 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:13 smithi057 ceph-mon[114331]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:13.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:13 smithi087 ceph-mon[109386]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:14.862 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T20:03:14.862 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 17s ago 23m - - 2023-10-01T20:03:14.862 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 18s ago 23m - - 2023-10-01T20:03:14.862 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (12m) 17s ago 19m 27.5M - 0.23.0 ba2b418f427c 0b16afb678d9 2023-10-01T20:03:14.862 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (11m) 18s ago 19m 58.2M - 8.3.5 dad864ee21e9 ca8ad8b1ccf8 2023-10-01T20:03:14.862 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (2m) 17s ago 19m 56.4M - 3.5 e1d6a67b021e ee01652ef7f1 2023-10-01T20:03:14.862 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283 running (14m) 18s ago 23m 519M - 18.0.0-6505-g37d71fd3 bea6d6056140 f7b2c168ac86 2023-10-01T20:03:14.863 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (9m) 17s ago 24m 451M - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T20:03:14.863 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (24m) 17s ago 24m 177M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T20:03:14.863 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (23m) 18s ago 23m 104M 2048M 17.2.0 e1d6a67b021e 4734e639b775 2023-10-01T20:03:14.863 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (23m) 17s ago 23m 94.5M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T20:03:14.863 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (12m) 17s ago 20m 24.3M - 1.3.1 1dbe0e931976 41af659015b7 2023-10-01T20:03:14.863 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (12m) 18s ago 20m 26.8M - 1.3.1 1dbe0e931976 dd38d7fd60ea 2023-10-01T20:03:14.863 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (22m) 17s ago 22m 157M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T20:03:14.863 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (22m) 17s ago 22m 165M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T20:03:14.863 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (22m) 17s ago 22m 122M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T20:03:14.863 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (21m) 17s ago 21m 180M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T20:03:14.863 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (21m) 18s ago 21m 173M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T20:03:14.863 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (21m) 18s ago 21m 166M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T20:03:14.864 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (21m) 18s ago 21m 134M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T20:03:14.864 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (20m) 18s ago 20m 162M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T20:03:14.864 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (2m) 18s ago 20m 75.5M - 2.33.4 514e6a882f6e 844c1a351b1e 2023-10-01T20:03:14.864 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (19m) 17s ago 19m 98.9M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T20:03:14.864 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (19m) 18s ago 19m 95.1M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T20:03:15.346 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch ls' 2023-10-01T20:03:16.335 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:16 smithi057 ceph-mon[108286]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:16.336 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:16 smithi057 ceph-mon[108286]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:16.336 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:16 smithi057 ceph-mon[114331]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:16.336 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:16 smithi057 ceph-mon[114331]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:16 smithi087 ceph-mon[109386]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:16 smithi087 ceph-mon[109386]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:17.298 INFO:teuthology.orchestra.run.smithi057.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-01T20:03:17.299 INFO:teuthology.orchestra.run.smithi057.stdout:agent 2/2 19s ago 3m * 2023-10-01T20:03:17.299 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager ?:9093,9094 1/1 19s ago 20m smithi057=a;count:1 2023-10-01T20:03:17.299 INFO:teuthology.orchestra.run.smithi057.stdout:grafana ?:3000 1/1 0s ago 20m smithi087=a;count:1 2023-10-01T20:03:17.299 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo ?:5000 1/1 19s ago 19m count:1 2023-10-01T20:03:17.299 INFO:teuthology.orchestra.run.smithi057.stdout:mgr 2/2 19s ago 23m smithi057=y;smithi087=x;count:2 2023-10-01T20:03:17.299 INFO:teuthology.orchestra.run.smithi057.stdout:mon 3/3 19s ago 23m smithi057:172.21.15.57=a;smithi057:[v2:172.21.15.57:3301,v1:172.21.15.57:6790]=c;smithi087:172.21.15.87=b;count:3 2023-10-01T20:03:17.299 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter ?:9100 2/2 19s ago 20m smithi057=a;smithi087=b;count:2 2023-10-01T20:03:17.300 INFO:teuthology.orchestra.run.smithi057.stdout:osd 8 19s ago - 2023-10-01T20:03:17.300 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus ?:9095 1/1 0s ago 20m smithi087=a;count:1 2023-10-01T20:03:17.300 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo ?:8000 2/2 19s ago 19m count:2 2023-10-01T20:03:18.010 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:17 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:18.010 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:17 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:18.010 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:17 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:18.010 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:17 smithi057 ceph-mon[108286]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:18.011 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:17 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:18.011 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:17 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:18.011 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:17 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:18.011 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:17 smithi057 ceph-mon[114331]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:18.037 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions' 2023-10-01T20:03:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:17 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:17 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:17 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:18.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:17 smithi087 ceph-mon[109386]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:19.110 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:18 smithi057 ceph-mon[108286]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:19.111 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:18 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:19.111 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:18 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:19.111 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:18 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:19.111 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:18 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:19.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:18 smithi057 ceph-mon[114331]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:19.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:18 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:19.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:18 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:19.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:18 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:19.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:18 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:19.317 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:18 smithi087 ceph-mon[109386]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:19.317 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:18 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:19.317 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:18 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:19.317 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:18 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:19.317 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:18 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:20.028 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:03:20.028 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2023-10-01T20:03:20.028 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2023-10-01T20:03:20.028 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:03:20.028 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2023-10-01T20:03:20.028 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:03:20.028 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:03:20.028 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2023-10-01T20:03:20.028 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2023-10-01T20:03:20.029 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:03:20.029 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": {}, 2023-10-01T20:03:20.029 INFO:teuthology.orchestra.run.smithi057.stdout: "rgw": { 2023-10-01T20:03:20.029 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T20:03:20.029 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:03:20.029 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2023-10-01T20:03:20.029 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 13, 2023-10-01T20:03:20.029 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:03:20.029 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T20:03:20.029 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:03:20.217 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:19 smithi057 ceph-mon[108286]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:20.217 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:19 smithi057 ceph-mon[114331]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:20.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:19 smithi087 ceph-mon[109386]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:20.422 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2023-10-01T20:03:21.010 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:03:20] "GET /metrics HTTP/1.1" 200 37903 "" "Prometheus/2.33.4" 2023-10-01T20:03:21.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:21 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/694878076' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:21.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:21 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:21.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:21 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:21.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:21 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:21.079 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:21 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:21.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:21 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/694878076' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:21.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:21 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:21.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:21 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:21.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:21 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:21.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:21 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:21.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:21 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/694878076' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:21 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:21 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:21 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:21 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:22 smithi087 ceph-mon[109386]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:22.442 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:22 smithi057 ceph-mon[108286]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:22.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:22 smithi057 ceph-mon[114331]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:23.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:23 smithi057 ceph-mon[108286]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:23.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:23 smithi057 ceph-mon[114331]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:23.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:23 smithi087 ceph-mon[109386]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:23.705 INFO:teuthology.orchestra.run.smithi057.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:03:24.111 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2023-10-01T20:03:24.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:24 smithi087 ceph-mon[109386]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:24 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:24 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:24 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:24 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:24 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:24.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:24 smithi057 ceph-mon[108286]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:24.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:24 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:24.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:24 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:24.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:24 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:24.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:24 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:24.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:24 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:24.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:24 smithi057 ceph-mon[114331]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:24.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:24 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:24.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:24 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:24.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:24 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:24.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:24 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:24.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:24 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:25.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:25 smithi057 ceph-mon[114331]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:03:25.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:25 smithi057 ceph-mon[114331]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:03:25.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:25 smithi057 ceph-mon[108286]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:03:25.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:25 smithi057 ceph-mon[108286]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:03:25.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:25 smithi087 ceph-mon[109386]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:03:25.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:25 smithi087 ceph-mon[109386]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:03:25.911 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:03:26.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:26 smithi057 ceph-mon[114331]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:26.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:26 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:26.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:26 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:26.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:26 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2023-10-01T20:03:26.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:26 smithi057 ceph-mon[114331]: osdmap e107: 8 total, 8 up, 8 in 2023-10-01T20:03:26.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:26 smithi057 ceph-mon[108286]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:26.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:26 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:26.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:26 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:26.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:26 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2023-10-01T20:03:26.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:26 smithi057 ceph-mon[108286]: osdmap e107: 8 total, 8 up, 8 in 2023-10-01T20:03:26.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:26 smithi087 ceph-mon[109386]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:26.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:26 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' 2023-10-01T20:03:26.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:26 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:26.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:26 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2023-10-01T20:03:26.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:26 smithi087 ceph-mon[109386]: osdmap e107: 8 total, 8 up, 8 in 2023-10-01T20:03:26.620 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T20:03:26.620 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 8s ago 23m - - 2023-10-01T20:03:26.620 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 9s ago 23m - - 2023-10-01T20:03:26.620 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (12m) 8s ago 20m 27.6M - 0.23.0 ba2b418f427c 0b16afb678d9 2023-10-01T20:03:26.620 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (12m) 9s ago 19m 58.2M - 8.3.5 dad864ee21e9 ca8ad8b1ccf8 2023-10-01T20:03:26.620 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (3m) 8s ago 19m 58.6M - 3.5 e1d6a67b021e ee01652ef7f1 2023-10-01T20:03:26.620 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283 running (14m) 9s ago 23m 520M - 18.0.0-6505-g37d71fd3 bea6d6056140 f7b2c168ac86 2023-10-01T20:03:26.620 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (9m) 8s ago 24m 452M - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T20:03:26.621 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (24m) 8s ago 24m 179M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T20:03:26.621 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (23m) 9s ago 23m 106M 2048M 17.2.0 e1d6a67b021e 4734e639b775 2023-10-01T20:03:26.621 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (23m) 8s ago 23m 95.7M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T20:03:26.621 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (12m) 8s ago 20m 24.3M - 1.3.1 1dbe0e931976 41af659015b7 2023-10-01T20:03:26.621 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (12m) 9s ago 20m 27.5M - 1.3.1 1dbe0e931976 dd38d7fd60ea 2023-10-01T20:03:26.621 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (23m) 8s ago 23m 158M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T20:03:26.621 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (22m) 8s ago 22m 166M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T20:03:26.621 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (22m) 8s ago 22m 123M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T20:03:26.621 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (22m) 8s ago 22m 182M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T20:03:26.621 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (21m) 9s ago 21m 174M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T20:03:26.621 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (21m) 9s ago 21m 168M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T20:03:26.621 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (21m) 9s ago 21m 135M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T20:03:26.622 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (21m) 9s ago 21m 164M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T20:03:26.622 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (3m) 9s ago 20m 75.5M - 2.33.4 514e6a882f6e 844c1a351b1e 2023-10-01T20:03:26.622 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (19m) 8s ago 19m 99.0M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T20:03:26.622 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (19m) 9s ago 19m 95.2M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T20:03:27.056 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:03:27.056 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2023-10-01T20:03:27.057 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2023-10-01T20:03:27.057 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:03:27.057 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2023-10-01T20:03:27.057 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 1 2023-10-01T20:03:27.057 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:03:27.057 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2023-10-01T20:03:27.057 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2023-10-01T20:03:27.057 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:03:27.057 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": {}, 2023-10-01T20:03:27.057 INFO:teuthology.orchestra.run.smithi057.stdout: "rgw": { 2023-10-01T20:03:27.057 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T20:03:27.058 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:03:27.058 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2023-10-01T20:03:27.058 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 13, 2023-10-01T20:03:27.058 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 1 2023-10-01T20:03:27.058 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T20:03:27.058 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:03:27.331 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ignoring --setuser ceph since I am not root 2023-10-01T20:03:27.331 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: ignoring --setgroup ceph since I am not root 2023-10-01T20:03:27.331 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mgr[137104]: -- 172.21.15.87:0/2114564405 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c69dc31e0 con 0x555c69d99000 2023-10-01T20:03:27.331 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:27.162+0000 7f20eb433200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T20:03:27.332 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:27.330+0000 7f20eb433200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T20:03:27.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:27.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: Upgrade: Target is version 18.0.0-6505-g37d71fd3 (reef) 2023-10-01T20:03:27.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24, digests ['quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24'] 2023-10-01T20:03:27.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: Upgrade: Need to upgrade myself (mgr.x) 2023-10-01T20:03:27.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: Upgrade: Need to upgrade myself (mgr.x) 2023-10-01T20:03:27.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: Failing over to other MGR 2023-10-01T20:03:27.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:27.332 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2023-10-01T20:03:27.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: mgrmap e38: y(active, starting, since 0.999296s) 2023-10-01T20:03:27.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:03:27.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:03:27.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:03:27.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T20:03:27.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:03:27.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T20:03:27.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T20:03:27.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T20:03:27.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T20:03:27.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T20:03:27.333 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T20:03:27.334 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T20:03:27.334 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T20:03:27.334 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T20:03:27.334 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T20:03:27.334 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-mon[109386]: Manager daemon y is now available 2023-10-01T20:03:27.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:03:26 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:03:26] ENGINE Bus STOPPING 2023-10-01T20:03:27.422 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:03:27] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-10-01T20:03:27.422 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:03:27] ENGINE Bus STOPPED 2023-10-01T20:03:27.422 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:03:27] ENGINE Bus STARTING 2023-10-01T20:03:27.422 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:03:27] ENGINE Serving on http://:::9283 2023-10-01T20:03:27.422 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:03:27] ENGINE Bus STARTED 2023-10-01T20:03:27.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: Upgrade: Target is version 18.0.0-6505-g37d71fd3 (reef) 2023-10-01T20:03:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24, digests ['quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24'] 2023-10-01T20:03:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: Upgrade: Need to upgrade myself (mgr.x) 2023-10-01T20:03:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: Upgrade: Need to upgrade myself (mgr.x) 2023-10-01T20:03:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: Failing over to other MGR 2023-10-01T20:03:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2023-10-01T20:03:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: mgrmap e38: y(active, starting, since 0.999296s) 2023-10-01T20:03:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:03:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:03:27.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:03:27.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T20:03:27.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:03:27.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T20:03:27.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T20:03:27.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T20:03:27.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T20:03:27.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T20:03:27.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T20:03:27.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T20:03:27.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T20:03:27.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T20:03:27.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T20:03:27.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[108286]: Manager daemon y is now available 2023-10-01T20:03:27.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:27.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: Upgrade: Target is version 18.0.0-6505-g37d71fd3 (reef) 2023-10-01T20:03:27.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24, digests ['quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24'] 2023-10-01T20:03:27.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: Upgrade: Need to upgrade myself (mgr.x) 2023-10-01T20:03:27.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: Upgrade: Need to upgrade myself (mgr.x) 2023-10-01T20:03:27.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: Failing over to other MGR 2023-10-01T20:03:27.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:27.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25103 172.21.15.87:0/4269422502' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2023-10-01T20:03:27.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: mgrmap e38: y(active, starting, since 0.999296s) 2023-10-01T20:03:27.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:03:27.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:03:27.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:03:27.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T20:03:27.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:03:27.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T20:03:27.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T20:03:27.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T20:03:27.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T20:03:27.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T20:03:27.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T20:03:27.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T20:03:27.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T20:03:27.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T20:03:27.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T20:03:27.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:27 smithi057 ceph-mon[114331]: Manager daemon y is now available 2023-10-01T20:03:28.001 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:03:28.002 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24", 2023-10-01T20:03:28.002 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2023-10-01T20:03:28.002 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2023-10-01T20:03:28.002 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2023-10-01T20:03:28.002 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2023-10-01T20:03:28.002 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-01T20:03:28.002 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "2/2 daemons upgraded", 2023-10-01T20:03:28.002 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "", 2023-10-01T20:03:28.002 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2023-10-01T20:03:28.002 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:03:28.055 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:27 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:27.656+0000 7f20eb433200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T20:03:28.376 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:28.151+0000 7f20eb433200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T20:03:28.377 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:28.377 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/3872131335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:28.377 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:28.377 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:28.377 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T20:03:28.377 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T20:03:28.377 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:28.377 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T20:03:28.378 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T20:03:28.378 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:03:28.378 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-mon[109386]: mgrmap e39: y(active, since 2s) 2023-10-01T20:03:28.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:28.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3872131335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:28.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:28.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:28.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T20:03:28.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T20:03:28.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:28.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T20:03:28.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T20:03:28.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:03:28.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[108286]: mgrmap e39: y(active, since 2s) 2023-10-01T20:03:28.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:28.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3872131335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:28.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:28.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:28.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T20:03:28.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T20:03:28.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:28.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T20:03:28.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T20:03:28.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:03:28.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:28 smithi057 ceph-mon[114331]: mgrmap e39: y(active, since 2s) 2023-10-01T20:03:28.633 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:28.376+0000 7f20eb433200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-01T20:03:29.059 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:28 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:28.797+0000 7f20eb433200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T20:03:29.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[108286]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:29.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[108286]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:29.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[108286]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:03:29.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:29.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:29.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:03:29.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:29.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:29.258 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[114331]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:29.258 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[114331]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:29.258 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[114331]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:03:29.259 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:29.259 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:29.259 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:03:29.259 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:29.259 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:29.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:29 smithi087 ceph-mon[109386]: from='client.25211 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:29 smithi087 ceph-mon[109386]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:29 smithi087 ceph-mon[109386]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:03:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:03:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:29.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:29.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:29 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:29.565+0000 7f20eb433200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T20:03:29.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:29 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:29.662+0000 7f20eb433200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T20:03:30.152 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: [01/Oct/2023:20:03:28] ENGINE Bus STARTING 2023-10-01T20:03:30.152 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: [01/Oct/2023:20:03:28] ENGINE Serving on http://172.21.15.57:8765 2023-10-01T20:03:30.152 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: [01/Oct/2023:20:03:28] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T20:03:30.152 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: [01/Oct/2023:20:03:28] ENGINE Bus STARTED 2023-10-01T20:03:30.152 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:03:30.152 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T20:03:30.152 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T20:03:30.152 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: mgrmap e40: y(active, since 3s) 2023-10-01T20:03:30.152 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.152 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.153 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.153 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.153 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.153 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.153 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.417 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:30.392+0000 7f20eb433200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T20:03:30.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: [01/Oct/2023:20:03:28] ENGINE Bus STARTING 2023-10-01T20:03:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: [01/Oct/2023:20:03:28] ENGINE Serving on http://172.21.15.57:8765 2023-10-01T20:03:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: [01/Oct/2023:20:03:28] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T20:03:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: [01/Oct/2023:20:03:28] ENGINE Bus STARTED 2023-10-01T20:03:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:03:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T20:03:30.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T20:03:30.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: mgrmap e40: y(active, since 3s) 2023-10-01T20:03:30.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: [01/Oct/2023:20:03:28] ENGINE Bus STARTING 2023-10-01T20:03:30.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: [01/Oct/2023:20:03:28] ENGINE Serving on http://172.21.15.57:8765 2023-10-01T20:03:30.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: [01/Oct/2023:20:03:28] ENGINE Serving on https://172.21.15.57:7150 2023-10-01T20:03:30.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: [01/Oct/2023:20:03:28] ENGINE Bus STARTED 2023-10-01T20:03:30.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:03:30.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T20:03:30.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T20:03:30.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: mgrmap e40: y(active, since 3s) 2023-10-01T20:03:30.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:30.826 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:30.486+0000 7f20eb433200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T20:03:30.826 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:30.580+0000 7f20eb433200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T20:03:30.827 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:30.825+0000 7f20eb433200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T20:03:31.089 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:30 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:30.928+0000 7f20eb433200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T20:03:31.089 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:31.035+0000 7f20eb433200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T20:03:31.364 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:31.291+0000 7f20eb433200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T20:03:31.365 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:03:31.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:03:31.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:03:31.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:03:31.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:03:31.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.368 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T20:03:31.368 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:31.368 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:03:31.368 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:03:31.368 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T20:03:31.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:31.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:03:31.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:03:31.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:03:31.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:03:31.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:03:31.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.370 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:31.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi057.zfdcts", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-10-01T20:03:31.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:31.614 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:31.401+0000 7f20eb433200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T20:03:31.874 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:31.613+0000 7f20eb433200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T20:03:31.874 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:31 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:31.708+0000 7f20eb433200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T20:03:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-mon[109386]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:03:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-mon[109386]: Reconfiguring iscsi.foo.smithi057.zfdcts (dependencies changed)... 2023-10-01T20:03:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-mon[109386]: Reconfiguring daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T20:03:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-mon[109386]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:03:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-mon[109386]: mgrmap e41: y(active, since 5s) 2023-10-01T20:03:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[108286]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:03:32.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:32.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[108286]: Reconfiguring iscsi.foo.smithi057.zfdcts (dependencies changed)... 2023-10-01T20:03:32.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[108286]: Reconfiguring daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T20:03:32.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[108286]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:03:32.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[108286]: mgrmap e41: y(active, since 5s) 2023-10-01T20:03:32.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[114331]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:03:32.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:32.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[114331]: Reconfiguring iscsi.foo.smithi057.zfdcts (dependencies changed)... 2023-10-01T20:03:32.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[114331]: Reconfiguring daemon iscsi.foo.smithi057.zfdcts on smithi057 2023-10-01T20:03:32.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[114331]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:03:32.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[114331]: mgrmap e41: y(active, since 5s) 2023-10-01T20:03:32.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:32.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:32.531+0000 7f20eb433200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T20:03:32.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:32.738+0000 7f20eb433200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T20:03:33.230 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:32 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:32.944+0000 7f20eb433200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T20:03:33.607 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:33.607 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:33.607 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:33.607 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:33 smithi057 ceph-mon[108286]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:03:33.607 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:33.608 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:33.608 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:33.608 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:33 smithi057 ceph-mon[114331]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:03:33.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:33 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:33.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:33 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:33.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:33 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:33.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:33 smithi087 ceph-mon[109386]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:03:33.948 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:33.654+0000 7f20eb433200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T20:03:33.949 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:33.857+0000 7f20eb433200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T20:03:34.200 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:34.055+0000 7f20eb433200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T20:03:34.200 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:34.184+0000 7f20eb433200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T20:03:34.200 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:20:03:34] ENGINE Bus STARTING 2023-10-01T20:03:34.560 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: CherryPy Checker: 2023-10-01T20:03:34.560 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: The Application mounted at '' has an empty config. 2023-10-01T20:03:34.560 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: 2023-10-01T20:03:34.560 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:20:03:34] ENGINE Serving on http://:::9283 2023-10-01T20:03:34.560 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[137080]: [01/Oct/2023:20:03:34] ENGINE Bus STARTED 2023-10-01T20:03:35.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-mon[109386]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T20:03:35.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-mon[109386]: Standby manager daemon x started 2023-10-01T20:03:35.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/446878242' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T20:03:35.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/446878242' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T20:03:35.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/446878242' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T20:03:35.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/446878242' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T20:03:35.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1045895436' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T20:03:35.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[114331]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T20:03:35.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[114331]: Standby manager daemon x started 2023-10-01T20:03:35.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/446878242' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T20:03:35.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/446878242' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T20:03:35.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/446878242' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T20:03:35.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/446878242' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T20:03:35.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1045895436' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T20:03:35.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[108286]: Reconfiguring prometheus.a (dependencies changed)... 2023-10-01T20:03:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[108286]: Standby manager daemon x started 2023-10-01T20:03:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/446878242' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T20:03:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/446878242' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T20:03:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/446878242' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T20:03:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/446878242' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T20:03:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1045895436' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-01T20:03:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:36.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-mon[109386]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T20:03:36.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-mon[109386]: mgrmap e42: y(active, since 8s), standbys: x 2023-10-01T20:03:36.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T20:03:36.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-mon[109386]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T20:03:36.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:36.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:36.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:36.134 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 systemd[1]: Stopping Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:03:36.134 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 bash[162074]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus.a" found: no such container 2023-10-01T20:03:36.134 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:03:35.909Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2023-10-01T20:03:36.135 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:03:35.910Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2023-10-01T20:03:36.135 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:03:35.910Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2023-10-01T20:03:36.135 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:03:35.910Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2023-10-01T20:03:36.135 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:03:35.910Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2023-10-01T20:03:36.135 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:03:35.910Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2023-10-01T20:03:36.135 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:03:35.910Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2023-10-01T20:03:36.135 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:03:35.910Z caller=main.go:828 level=info msg="Scrape manager stopped" 2023-10-01T20:03:36.135 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:03:35.910Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2023-10-01T20:03:36.135 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:03:35.910Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2023-10-01T20:03:36.135 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:03:35.910Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2023-10-01T20:03:36.135 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:35 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[156879]: ts=2023-10-01T20:03:35.910Z caller=main.go:1066 level=info msg="See you next time!" 2023-10-01T20:03:36.135 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 bash[162087]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a 2023-10-01T20:03:36.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[108286]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T20:03:36.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[108286]: mgrmap e42: y(active, since 8s), standbys: x 2023-10-01T20:03:36.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T20:03:36.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[108286]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T20:03:36.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:36.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:36.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:36.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[114331]: Reconfiguring daemon prometheus.a on smithi087 2023-10-01T20:03:36.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[114331]: mgrmap e42: y(active, since 8s), standbys: x 2023-10-01T20:03:36.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T20:03:36.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[114331]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T20:03:36.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:36.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:36.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:36.444 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 bash[162126]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus.a" found: no such container 2023-10-01T20:03:36.445 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@prometheus.a.service: Succeeded. 2023-10-01T20:03:36.445 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 systemd[1]: Stopped Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:03:36.445 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 systemd[1]: Starting Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:03:36.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.550Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2023-10-01T20:03:36.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.550Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2023-10-01T20:03:36.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.550Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2023-10-01T20:03:36.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.550Z caller=main.go:518 level=info host_details="(Linux 4.18.0-514.el8.x86_64 #1 SMP Tue Sep 19 13:19:58 UTC 2023 x86_64 smithi087 (none))" 2023-10-01T20:03:36.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.550Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2023-10-01T20:03:36.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.550Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2023-10-01T20:03:36.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.552Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2023-10-01T20:03:36.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.552Z caller=main.go:923 level=info msg="Starting TSDB ..." 2023-10-01T20:03:36.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.553Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2023-10-01T20:03:36.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 bash[162188]: 488484c340659a351d5c89efd53eaa617d0c5ed7a9f5870d623cecf8365cdfa1 2023-10-01T20:03:36.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.555Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2023-10-01T20:03:36.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.556Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=835.905µs 2023-10-01T20:03:36.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.556Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2023-10-01T20:03:36.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.557Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=5 2023-10-01T20:03:36.884 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 systemd[1]: Started Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:03:37.296 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:36 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:36.995Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=5 2023-10-01T20:03:37.297 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:37.037Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=5 2023-10-01T20:03:37.297 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:37.088Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=5 2023-10-01T20:03:37.297 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:37.096Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=4 maxSegment=5 2023-10-01T20:03:37.297 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:37.096Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=5 maxSegment=5 2023-10-01T20:03:37.297 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:37.096Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=66.157µs wal_replay_duration=540.010709ms total_replay_duration=540.929029ms 2023-10-01T20:03:37.297 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:37.100Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2023-10-01T20:03:37.297 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:37.100Z caller=main.go:947 level=info msg="TSDB started" 2023-10-01T20:03:37.297 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:37.100Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2023-10-01T20:03:37.297 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:37.115Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.648694ms db_storage=572ns remote_storage=1.614µs web_handler=397ns query_engine=593ns scrape=472.743µs scrape_sd=103.359µs notify=32.555µs notify_sd=15.567µs rules=13.68534ms 2023-10-01T20:03:37.297 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:03:37.115Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2023-10-01T20:03:37.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:37.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:37.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T20:03:37.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T20:03:37.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:37.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: Adding iSCSI gateway http://:@172.21.15.57:5000 to Dashboard 2023-10-01T20:03:37.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T20:03:37.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T20:03:37.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T20:03:37.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T20:03:37.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:37.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:03:37.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:03:37.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:37.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:37 smithi087 ceph-mon[109386]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2023-10-01T20:03:37.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:37.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T20:03:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T20:03:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: Adding iSCSI gateway http://:@172.21.15.57:5000 to Dashboard 2023-10-01T20:03:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T20:03:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T20:03:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T20:03:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T20:03:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:03:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:03:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[108286]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2023-10-01T20:03:37.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:37.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:37.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T20:03:37.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2023-10-01T20:03:37.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:37.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: Adding iSCSI gateway http://:@172.21.15.57:5000 to Dashboard 2023-10-01T20:03:37.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T20:03:37.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2023-10-01T20:03:37.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T20:03:37.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi057"}]: dispatch 2023-10-01T20:03:37.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:37.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:03:37.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:03:37.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:37.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:37 smithi057 ceph-mon[114331]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2023-10-01T20:03:38.691 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:38 smithi087 ceph-mon[109386]: Upgrade: Updating mgr.x 2023-10-01T20:03:38.691 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:03:38.691 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T20:03:38.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:38.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:38 smithi087 ceph-mon[109386]: Deploying daemon mgr.x on smithi087 2023-10-01T20:03:38.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:38 smithi057 ceph-mon[108286]: Upgrade: Updating mgr.x 2023-10-01T20:03:38.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:03:38.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T20:03:38.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:38.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:38 smithi057 ceph-mon[108286]: Deploying daemon mgr.x on smithi087 2023-10-01T20:03:38.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:38 smithi057 ceph-mon[114331]: Upgrade: Updating mgr.x 2023-10-01T20:03:38.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:03:38.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T20:03:38.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:38.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:38 smithi057 ceph-mon[114331]: Deploying daemon mgr.x on smithi087 2023-10-01T20:03:39.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:38 smithi087 systemd[1]: Stopping Ceph mgr.x for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:03:39.463 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:39 smithi087 bash[162619]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x 2023-10-01T20:03:39.463 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:39 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.x.service: Main process exited, code=exited, status=143/n/a 2023-10-01T20:03:39.798 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:39 smithi087 ceph-mon[109386]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T20:03:39.798 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:39 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.x.service: Failed with result 'exit-code'. 2023-10-01T20:03:39.798 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:39 smithi087 systemd[1]: Stopped Ceph mgr.x for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:03:39.798 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:39 smithi087 systemd[1]: Starting Ceph mgr.x for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:03:40.099 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:39 smithi087 ceph-mgr[162810]: -- 172.21.15.87:0/1579130736 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555e71ba11e0 con 0x555e71b76800 2023-10-01T20:03:40.099 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:39 smithi087 bash[162772]: b5797dd6a1b2bdd89f641b7621f16378c3247b9b1bb24cc584f334044f20f16b 2023-10-01T20:03:40.099 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:39 smithi087 systemd[1]: Started Ceph mgr.x for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:03:40.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:39 smithi057 ceph-mon[108286]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T20:03:40.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:39 smithi057 ceph-mon[114331]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T20:03:40.382 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:40.097+0000 7f18c7bdb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T20:03:40.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:40.269+0000 7f18c7bdb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T20:03:40.882 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:40 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:40.607+0000 7f18c7bdb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T20:03:41.052 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.052 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.052 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.052 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.052 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:41.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2023-10-01T20:03:41.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2023-10-01T20:03:41.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2023-10-01T20:03:41.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2023-10-01T20:03:41.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.173 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:41.171+0000 7f18c7bdb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T20:03:41.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2023-10-01T20:03:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2023-10-01T20:03:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2023-10-01T20:03:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2023-10-01T20:03:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2023-10-01T20:03:41.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2023-10-01T20:03:41.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2023-10-01T20:03:41.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2023-10-01T20:03:41.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.436 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.436 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.436 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:41.436 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:41.436 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:41.436 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:41.436 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:41 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:41.633 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:41.422+0000 7f18c7bdb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-01T20:03:42.054 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:41 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:41.910+0000 7f18c7bdb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T20:03:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: Upgrade: Setting container_image for all mgr 2023-10-01T20:03:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T20:03:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all mgr 2023-10-01T20:03:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T20:03:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all mgr 2023-10-01T20:03:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T20:03:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:42.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:42.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:42.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:42.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:42.983 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:42.733+0000 7f18c7bdb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T20:03:42.983 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:42 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:42.832+0000 7f18c7bdb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T20:03:43.317 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.317 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.318 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.318 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.318 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.318 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.318 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.318 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.318 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.318 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.318 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:43.319 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.319 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.319 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.319 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.319 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.319 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.319 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.319 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.319 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.319 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.320 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.320 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.320 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.320 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.320 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.320 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.320 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.320 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.320 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.321 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.321 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.321 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.321 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.321 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.321 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.321 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.321 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.321 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.321 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.322 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.322 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.322 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.322 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.322 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.322 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.322 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.322 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.322 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:43.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:43.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:43.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:43.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:43.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:43.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:43.609 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:43.508+0000 7f18c7bdb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T20:03:43.882 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:43.608+0000 7f18c7bdb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T20:03:43.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:43.708+0000 7f18c7bdb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T20:03:44.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T20:03:44.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.249 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:43.966+0000 7f18c7bdb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T20:03:44.249 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:44.076+0000 7f18c7bdb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T20:03:44.249 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:44.185+0000 7f18c7bdb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T20:03:44.249 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.249 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.249 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.249 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.249 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.250 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.250 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.250 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.250 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.250 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.250 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.250 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.250 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.250 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.250 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.250 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.251 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.251 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T20:03:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T20:03:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.435 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.435 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.435 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.435 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.435 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.436 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.436 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.436 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.436 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.436 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:44.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:44.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:44.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:44.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:44.779 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:44.442+0000 7f18c7bdb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T20:03:44.780 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:44.561+0000 7f18c7bdb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T20:03:44.780 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:44.779+0000 7f18c7bdb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T20:03:45.060 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:03:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:03:44] "GET /metrics HTTP/1.1" 200 37899 "" "Prometheus/2.33.4" 2023-10-01T20:03:45.062 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:44 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:44.883+0000 7f18c7bdb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T20:03:45.062 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.062 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.063 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.063 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.063 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.063 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.063 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:45.063 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.362 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.362 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.363 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.363 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.363 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.363 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.363 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.363 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.363 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.363 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.364 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.364 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.364 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.364 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.364 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.364 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.364 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.365 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.365 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.365 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.365 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.365 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.365 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.365 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.365 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.365 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.366 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.367 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:45.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:45.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:45.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:45.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:45.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:45.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:45.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:45.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:45 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.066 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:45.756+0000 7f18c7bdb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T20:03:46.066 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:45 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:45.967+0000 7f18c7bdb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T20:03:46.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:46.162+0000 7f18c7bdb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T20:03:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T20:03:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T20:03:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:46.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:46.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T20:03:46.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:46.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:46 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:46.922 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:46 smithi057 systemd[1]: Stopping Ceph node-exporter.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:03:47.129 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-mon[109386]: Upgrade: Setting container_image for all crash 2023-10-01T20:03:47.129 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-mon[109386]: Upgrade: Setting container_image for all mds 2023-10-01T20:03:47.129 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-mon[109386]: Upgrade: Setting container_image for all rbd-mirror 2023-10-01T20:03:47.129 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-mon[109386]: Upgrade: Setting container_image for all cephfs-mirror 2023-10-01T20:03:47.129 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-mon[109386]: Upgrade: Setting container_image for all ceph-exporter 2023-10-01T20:03:47.129 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-mon[109386]: Upgrade: Setting container_image for all nfs 2023-10-01T20:03:47.130 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-mon[109386]: Upgrade: Setting container_image for all nvmeof 2023-10-01T20:03:47.130 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:46 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:46.873+0000 7f18c7bdb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T20:03:47.130 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:47.075+0000 7f18c7bdb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T20:03:47.316 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:47 smithi057 bash[183682]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a 2023-10-01T20:03:47.316 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:47 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2023-10-01T20:03:47.316 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all crash 2023-10-01T20:03:47.316 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all mds 2023-10-01T20:03:47.316 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all rbd-mirror 2023-10-01T20:03:47.316 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all cephfs-mirror 2023-10-01T20:03:47.317 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all ceph-exporter 2023-10-01T20:03:47.317 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all nfs 2023-10-01T20:03:47.317 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all nvmeof 2023-10-01T20:03:47.317 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all crash 2023-10-01T20:03:47.317 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all mds 2023-10-01T20:03:47.317 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all rbd-mirror 2023-10-01T20:03:47.317 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all cephfs-mirror 2023-10-01T20:03:47.317 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all ceph-exporter 2023-10-01T20:03:47.318 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all nfs 2023-10-01T20:03:47.318 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:47 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all nvmeof 2023-10-01T20:03:47.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:47.279+0000 7f18c7bdb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T20:03:47.671 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:47 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.a.service: Failed with result 'exit-code'. 2023-10-01T20:03:47.671 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:47 smithi057 systemd[1]: Stopped Ceph node-exporter.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:03:47.672 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:47 smithi057 systemd[1]: Starting Ceph node-exporter.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:03:47.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:47.401+0000 7f18c7bdb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T20:03:47.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: [01/Oct/2023:20:03:47] ENGINE Bus STARTING 2023-10-01T20:03:47.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: CherryPy Checker: 2023-10-01T20:03:47.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: The Application mounted at '' has an empty config. 2023-10-01T20:03:47.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:03:47.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: [01/Oct/2023:20:03:47] ENGINE Serving on http://:::9283 2023-10-01T20:03:47.883 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:03:47 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: [01/Oct/2023:20:03:47] ENGINE Bus STARTED 2023-10-01T20:03:48.077 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:47 smithi057 bash[183832]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2023-10-01T20:03:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:48 smithi087 ceph-mon[109386]: Upgrade: Updating node-exporter.a (1/2) 2023-10-01T20:03:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:48 smithi087 ceph-mon[109386]: Deploying daemon node-exporter.a on smithi057 2023-10-01T20:03:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:48 smithi087 ceph-mon[109386]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:48 smithi087 ceph-mon[109386]: Standby manager daemon x restarted 2023-10-01T20:03:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:48 smithi087 ceph-mon[109386]: Standby manager daemon x started 2023-10-01T20:03:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:48 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/1590191823' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T20:03:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:48 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/1590191823' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T20:03:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:48 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/1590191823' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T20:03:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:48 smithi087 ceph-mon[109386]: from='mgr.? 172.21.15.87:0/1590191823' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T20:03:48.421 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:48 smithi057 bash[183832]: Getting image source signatures 2023-10-01T20:03:48.422 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:48 smithi057 bash[183832]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2023-10-01T20:03:48.422 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:48 smithi057 bash[183832]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2023-10-01T20:03:48.422 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:48 smithi057 bash[183832]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2023-10-01T20:03:48.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[108286]: Upgrade: Updating node-exporter.a (1/2) 2023-10-01T20:03:48.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[108286]: Deploying daemon node-exporter.a on smithi057 2023-10-01T20:03:48.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[108286]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:48.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[108286]: Standby manager daemon x restarted 2023-10-01T20:03:48.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[108286]: Standby manager daemon x started 2023-10-01T20:03:48.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/1590191823' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T20:03:48.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/1590191823' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T20:03:48.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/1590191823' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T20:03:48.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[108286]: from='mgr.? 172.21.15.87:0/1590191823' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T20:03:48.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[114331]: Upgrade: Updating node-exporter.a (1/2) 2023-10-01T20:03:48.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[114331]: Deploying daemon node-exporter.a on smithi057 2023-10-01T20:03:48.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[114331]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:48.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[114331]: Standby manager daemon x restarted 2023-10-01T20:03:48.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[114331]: Standby manager daemon x started 2023-10-01T20:03:48.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/1590191823' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T20:03:48.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/1590191823' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T20:03:48.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/1590191823' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T20:03:48.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:48 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/1590191823' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T20:03:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:49 smithi087 ceph-mon[109386]: mgrmap e43: y(active, since 22s), standbys: x 2023-10-01T20:03:49.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-mon[108286]: mgrmap e43: y(active, since 22s), standbys: x 2023-10-01T20:03:49.416 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-mon[114331]: mgrmap e43: y(active, since 22s), standbys: x 2023-10-01T20:03:49.416 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 bash[183832]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2023-10-01T20:03:49.416 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 bash[183832]: Writing manifest to image destination 2023-10-01T20:03:50.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:50 smithi087 ceph-mon[109386]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:50.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:50 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:50.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:50 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:50.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:50 smithi057 ceph-mon[108286]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:50.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:50 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:50.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:50 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:50.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:50 smithi057 ceph-mon[114331]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:50 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:50 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:50.423 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.991Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2023-10-01T20:03:50.423 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.991Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2023-10-01T20:03:50.423 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.991Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2023-10-01T20:03:50.424 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.991Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2023-10-01T20:03:50.424 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.992Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2023-10-01T20:03:50.424 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.992Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2023-10-01T20:03:50.424 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2023-10-01T20:03:50.424 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=arp 2023-10-01T20:03:50.424 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=bcache 2023-10-01T20:03:50.424 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=bonding 2023-10-01T20:03:50.424 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=btrfs 2023-10-01T20:03:50.424 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=conntrack 2023-10-01T20:03:50.425 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=cpu 2023-10-01T20:03:50.425 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=cpufreq 2023-10-01T20:03:50.425 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=diskstats 2023-10-01T20:03:50.425 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=dmi 2023-10-01T20:03:50.425 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=edac 2023-10-01T20:03:50.425 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=entropy 2023-10-01T20:03:50.425 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=fibrechannel 2023-10-01T20:03:50.426 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=filefd 2023-10-01T20:03:50.426 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=filesystem 2023-10-01T20:03:50.426 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=hwmon 2023-10-01T20:03:50.426 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=infiniband 2023-10-01T20:03:50.426 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=ipvs 2023-10-01T20:03:50.426 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=loadavg 2023-10-01T20:03:50.426 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=mdadm 2023-10-01T20:03:50.426 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=meminfo 2023-10-01T20:03:50.426 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=netclass 2023-10-01T20:03:50.426 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=netdev 2023-10-01T20:03:50.427 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=netstat 2023-10-01T20:03:50.427 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=nfs 2023-10-01T20:03:50.427 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=nfsd 2023-10-01T20:03:50.427 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=nvme 2023-10-01T20:03:50.427 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=os 2023-10-01T20:03:50.427 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2023-10-01T20:03:50.427 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=pressure 2023-10-01T20:03:50.427 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=rapl 2023-10-01T20:03:50.427 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=schedstat 2023-10-01T20:03:50.427 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=selinux 2023-10-01T20:03:50.427 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=sockstat 2023-10-01T20:03:50.428 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=softnet 2023-10-01T20:03:50.428 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=stat 2023-10-01T20:03:50.428 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=tapestats 2023-10-01T20:03:50.428 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=textfile 2023-10-01T20:03:50.428 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=thermal_zone 2023-10-01T20:03:50.428 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=time 2023-10-01T20:03:50.428 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=udp_queues 2023-10-01T20:03:50.428 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=uname 2023-10-01T20:03:50.428 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=vmstat 2023-10-01T20:03:50.428 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=xfs 2023-10-01T20:03:50.429 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.993Z caller=node_exporter.go:117 level=info collector=zfs 2023-10-01T20:03:50.429 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.994Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2023-10-01T20:03:50.429 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:49 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a[183882]: ts=2023-10-01T20:03:49.994Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2023-10-01T20:03:50.429 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:50 smithi057 bash[183832]: 175d4990d5fdebf462ece1e7b92e154b354342df5f1246784e2c95da4ae3a053 2023-10-01T20:03:50.429 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:03:50 smithi057 systemd[1]: Started Ceph node-exporter.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:03:51.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:51 smithi087 ceph-mon[109386]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:51.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:51 smithi057 ceph-mon[108286]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:51.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:51 smithi057 ceph-mon[114331]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:51.871 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:51 smithi087 systemd[1]: Stopping Ceph node-exporter.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:03:52.122 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:51 smithi087 bash[165706]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b 2023-10-01T20:03:52.123 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:52 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2023-10-01T20:03:52.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:52 smithi057 ceph-mon[108286]: Upgrade: Updating node-exporter.b (2/2) 2023-10-01T20:03:52.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:52 smithi057 ceph-mon[108286]: Deploying daemon node-exporter.b on smithi087 2023-10-01T20:03:52.379 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:52 smithi057 ceph-mon[114331]: Upgrade: Updating node-exporter.b (2/2) 2023-10-01T20:03:52.379 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:52 smithi057 ceph-mon[114331]: Deploying daemon node-exporter.b on smithi087 2023-10-01T20:03:52.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:52 smithi087 ceph-mon[109386]: Upgrade: Updating node-exporter.b (2/2) 2023-10-01T20:03:52.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:52 smithi087 ceph-mon[109386]: Deploying daemon node-exporter.b on smithi087 2023-10-01T20:03:52.383 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:52 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.b.service: Failed with result 'exit-code'. 2023-10-01T20:03:52.383 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:52 smithi087 systemd[1]: Stopped Ceph node-exporter.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:03:52.383 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:52 smithi087 systemd[1]: Starting Ceph node-exporter.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:03:52.844 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:52 smithi087 bash[165854]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2023-10-01T20:03:53.129 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:52 smithi087 bash[165854]: Getting image source signatures 2023-10-01T20:03:53.129 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:52 smithi087 bash[165854]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2023-10-01T20:03:53.129 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:52 smithi087 bash[165854]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2023-10-01T20:03:53.129 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:52 smithi087 bash[165854]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2023-10-01T20:03:53.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:53 smithi087 ceph-mon[109386]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:53.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:53 smithi057 ceph-mon[108286]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:53.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:53 smithi057 ceph-mon[114331]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:54.133 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:53 smithi087 bash[165854]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2023-10-01T20:03:54.133 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:53 smithi087 bash[165854]: Writing manifest to image destination 2023-10-01T20:03:54.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:54 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:54.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:54 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:54.428 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-mon[109386]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:03:54.884 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.428Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2023-10-01T20:03:54.884 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.428Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2023-10-01T20:03:54.884 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2023-10-01T20:03:54.884 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2023-10-01T20:03:54.884 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2023-10-01T20:03:54.884 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2023-10-01T20:03:54.884 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2023-10-01T20:03:54.884 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=arp 2023-10-01T20:03:54.884 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=bcache 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=bonding 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=btrfs 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=conntrack 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=cpu 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=cpufreq 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=diskstats 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=dmi 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=edac 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=entropy 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=fibrechannel 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=filefd 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=filesystem 2023-10-01T20:03:54.885 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.429Z caller=node_exporter.go:117 level=info collector=hwmon 2023-10-01T20:03:54.886 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=infiniband 2023-10-01T20:03:54.886 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=ipvs 2023-10-01T20:03:54.886 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=loadavg 2023-10-01T20:03:54.886 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=mdadm 2023-10-01T20:03:54.886 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=meminfo 2023-10-01T20:03:54.886 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=netclass 2023-10-01T20:03:54.886 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=netdev 2023-10-01T20:03:54.886 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=netstat 2023-10-01T20:03:54.886 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=nfs 2023-10-01T20:03:54.886 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=nfsd 2023-10-01T20:03:54.886 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=nvme 2023-10-01T20:03:54.887 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=os 2023-10-01T20:03:54.887 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2023-10-01T20:03:54.887 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=pressure 2023-10-01T20:03:54.887 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=rapl 2023-10-01T20:03:54.887 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=schedstat 2023-10-01T20:03:54.887 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=selinux 2023-10-01T20:03:54.887 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=sockstat 2023-10-01T20:03:54.887 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=softnet 2023-10-01T20:03:54.887 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=stat 2023-10-01T20:03:54.888 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=tapestats 2023-10-01T20:03:54.888 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=textfile 2023-10-01T20:03:54.888 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=thermal_zone 2023-10-01T20:03:54.888 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=time 2023-10-01T20:03:54.888 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=udp_queues 2023-10-01T20:03:54.888 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=uname 2023-10-01T20:03:54.888 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=vmstat 2023-10-01T20:03:54.888 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=xfs 2023-10-01T20:03:54.888 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=node_exporter.go:117 level=info collector=zfs 2023-10-01T20:03:54.889 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2023-10-01T20:03:54.889 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b[165904]: ts=2023-10-01T20:03:54.430Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2023-10-01T20:03:54.889 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 bash[165854]: b2605eeb2dd46b3b28defb3a8adaf1203a658fb7a122567e9e8543d188259af5 2023-10-01T20:03:54.889 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:03:54 smithi087 systemd[1]: Started Ceph node-exporter.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:03:55.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:03:54 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:03:54] "GET /metrics HTTP/1.1" 200 37899 "" "Prometheus/2.33.4" 2023-10-01T20:03:55.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:55.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:55.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.825 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.825 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.825 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.825 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.826 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.826 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.826 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.826 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.826 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.826 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.826 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:55.826 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.827 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.827 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.827 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.827 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.827 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.827 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:55.827 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.827 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.827 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.827 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.827 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.827 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:55.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:55.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:55.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.829 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.830 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.830 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.830 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:55 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:55.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:55.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:55.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:03:55.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:55.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.876 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.876 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.876 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:55.876 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.876 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:55.876 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:55.876 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:55.876 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:55.876 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:55 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:56 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:56.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:56.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:56.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:56.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:56.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:56.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:56 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:57.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:57.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:57 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:57.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:57.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:03:57.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:57.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:03:57.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:57.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:57.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:57.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:57.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:57.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:57 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.393 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:03:58.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:58.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:58.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:58.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:58.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:58.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:58.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.152 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T20:03:59.152 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 2s ago 24m - - 2023-10-01T20:03:59.152 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 13s ago 24m - - 2023-10-01T20:03:59.152 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (13m) 2s ago 20m 27.6M - 0.23.0 ba2b418f427c 0b16afb678d9 2023-10-01T20:03:59.152 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (12m) 13s ago 20m 57.8M - 8.3.5 dad864ee21e9 ca8ad8b1ccf8 2023-10-01T20:03:59.152 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (25s) 2s ago 19m 52.1M - 3.5 e1d6a67b021e 7ef2818f7c68 2023-10-01T20:03:59.152 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283,8765 running (19s) 13s ago 23m 328M - 18.0.0-6505-g37d71fd3 bea6d6056140 b5797dd6a1b2 2023-10-01T20:03:59.153 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (10m) 2s ago 25m 511M - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T20:03:59.153 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (25m) 2s ago 25m 193M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T20:03:59.153 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (24m) 13s ago 24m 116M 2048M 17.2.0 e1d6a67b021e 4734e639b775 2023-10-01T20:03:59.153 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (24m) 2s ago 24m 100M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T20:03:59.153 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (9s) 2s ago 20m 7868k - 1.5.0 0da6a335fe13 175d4990d5fd 2023-10-01T20:03:59.153 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 starting - - - - 2023-10-01T20:03:59.153 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (23m) 2s ago 23m 163M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T20:03:59.153 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (23m) 2s ago 23m 171M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T20:03:59.153 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (22m) 2s ago 22m 125M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T20:03:59.153 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (22m) 2s ago 22m 189M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T20:03:59.153 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (22m) 13s ago 22m 177M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T20:03:59.153 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (22m) 13s ago 22m 170M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T20:03:59.154 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (21m) 13s ago 21m 137M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T20:03:59.154 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (21m) 13s ago 21m 167M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T20:03:59.154 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (22s) 13s ago 21m 48.5M - 2.33.4 514e6a882f6e 488484c34065 2023-10-01T20:03:59.154 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (20m) 2s ago 20m 99.8M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T20:03:59.154 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (20m) 13s ago 20m 95.5M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T20:03:59.594 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:03:59.594 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2023-10-01T20:03:59.594 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2023-10-01T20:03:59.594 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:03:59.594 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2023-10-01T20:03:59.594 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:03:59.594 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:03:59.594 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2023-10-01T20:03:59.594 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2023-10-01T20:03:59.594 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:03:59.595 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": {}, 2023-10-01T20:03:59.595 INFO:teuthology.orchestra.run.smithi057.stdout: "rgw": { 2023-10-01T20:03:59.595 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T20:03:59.595 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:03:59.595 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2023-10-01T20:03:59.595 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 13, 2023-10-01T20:03:59.595 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:03:59.595 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T20:03:59.595 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:03:59.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='client.25339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:59.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T20:03:59.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.873 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.874 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.875 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:03:59 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='client.25339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:59.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T20:03:59.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='client.25339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:59.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:03:59.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T20:03:59.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:03:59.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:03:59.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:03:59.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:03:59.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:03:59.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:03:59 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.007 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:04:00.008 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24", 2023-10-01T20:04:00.008 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2023-10-01T20:04:00.008 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2023-10-01T20:04:00.008 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2023-10-01T20:04:00.008 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2023-10-01T20:04:00.008 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-01T20:04:00.008 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "2/2 daemons upgraded", 2023-10-01T20:04:00.008 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading prometheus daemons", 2023-10-01T20:04:00.008 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2023-10-01T20:04:00.008 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:04:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/120988081' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:00 smithi087 ceph-mon[109386]: from='client.25363 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:00.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:00.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:00.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:00.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:00.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:00.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/120988081' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:00.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:00.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[108286]: from='client.25363 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:00.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:00.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:00.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:00.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/120988081' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:00.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:00.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:00.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:00 smithi057 ceph-mon[114331]: from='client.25363 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:01 smithi087 ceph-mon[109386]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T20:04:01.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:01 smithi057 ceph-mon[108286]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T20:04:01.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:01 smithi057 ceph-mon[114331]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T20:04:02.878 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:02 smithi087 ceph-mon[109386]: Upgrade: Updating prometheus.a 2023-10-01T20:04:02.878 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:02 smithi087 ceph-mon[109386]: Deploying daemon prometheus.a on smithi087 2023-10-01T20:04:02.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:02 smithi057 ceph-mon[108286]: Upgrade: Updating prometheus.a 2023-10-01T20:04:02.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:02 smithi057 ceph-mon[108286]: Deploying daemon prometheus.a on smithi087 2023-10-01T20:04:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:02 smithi057 ceph-mon[114331]: Upgrade: Updating prometheus.a 2023-10-01T20:04:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:02 smithi057 ceph-mon[114331]: Deploying daemon prometheus.a on smithi087 2023-10-01T20:04:03.849 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:03 smithi087 ceph-mon[109386]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T20:04:03.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:03 smithi057 ceph-mon[108286]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T20:04:03.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:03 smithi057 ceph-mon[114331]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T20:04:04.133 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:03 smithi087 systemd[1]: Stopping Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:04:04.598 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:04:04.173Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2023-10-01T20:04:04.598 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:04:04.173Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2023-10-01T20:04:04.598 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:04:04.173Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2023-10-01T20:04:04.598 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:04:04.173Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2023-10-01T20:04:04.598 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:04:04.173Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2023-10-01T20:04:04.598 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:04:04.173Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2023-10-01T20:04:04.599 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:04:04.173Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2023-10-01T20:04:04.599 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:04:04.173Z caller=main.go:828 level=info msg="Scrape manager stopped" 2023-10-01T20:04:04.599 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:04:04.173Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2023-10-01T20:04:04.599 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:04:04.174Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2023-10-01T20:04:04.599 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:04:04.174Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2023-10-01T20:04:04.599 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[162202]: ts=2023-10-01T20:04:04.174Z caller=main.go:1066 level=info msg="See you next time!" 2023-10-01T20:04:04.599 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 bash[168936]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a 2023-10-01T20:04:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:04 smithi087 ceph-mon[109386]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:04.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@prometheus.a.service: Succeeded. 2023-10-01T20:04:04.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 systemd[1]: Stopped Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:04:04.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:04 smithi087 systemd[1]: Starting Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:04:04.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:04 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:04.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:04 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:05.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.098Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2023-10-01T20:04:05.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.098Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2023-10-01T20:04:05.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.098Z caller=main.go:570 level=info host_details="(Linux 4.18.0-514.el8.x86_64 #1 SMP Tue Sep 19 13:19:58 UTC 2023 x86_64 smithi087 (none))" 2023-10-01T20:04:05.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.098Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2023-10-01T20:04:05.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.098Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2023-10-01T20:04:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.099Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2023-10-01T20:04:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.100Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2023-10-01T20:04:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 bash[169086]: 79b0216ed806675e7d99fcb2b3d4dc3b84651c1ce26fbedcda880ea4e634c71a 2023-10-01T20:04:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.100Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2023-10-01T20:04:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.100Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2023-10-01T20:04:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 systemd[1]: Started Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:04:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.104Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2023-10-01T20:04:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.104Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=896.811µs 2023-10-01T20:04:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.104Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2023-10-01T20:04:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.105Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=6 2023-10-01T20:04:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.125Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=6 2023-10-01T20:04:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.130Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=6 2023-10-01T20:04:05.385 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.138Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=6 2023-10-01T20:04:05.385 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.140Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=4 maxSegment=6 2023-10-01T20:04:05.385 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.141Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=5 maxSegment=6 2023-10-01T20:04:05.385 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.141Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=6 maxSegment=6 2023-10-01T20:04:05.385 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.141Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=35.245µs wal_replay_duration=36.797557ms wbl_replay_duration=145ns total_replay_duration=37.753813ms 2023-10-01T20:04:05.385 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.145Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2023-10-01T20:04:05.385 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.145Z caller=main.go:1029 level=info msg="TSDB started" 2023-10-01T20:04:05.385 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.145Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2023-10-01T20:04:05.385 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.160Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.30774ms db_storage=844ns remote_storage=1.073µs web_handler=369ns query_engine=737ns scrape=578.488µs scrape_sd=108.684µs notify=15.522µs notify_sd=17.646µs rules=14.24816ms tracing=5.757µs 2023-10-01T20:04:05.385 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.160Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2023-10-01T20:04:05.385 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:05.160Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2023-10-01T20:04:06.286 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T20:04:06.286 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.286 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:04:06.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.288 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.289 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.289 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.289 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.289 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.289 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.289 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.289 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.289 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.289 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.289 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:06 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T20:04:06.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:04:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T20:04:06.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:04:06.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:06.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:06.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:06.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:06.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:06.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:06 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:04:07.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:07 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:04:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-10-01T20:04:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:07.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:07.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:07.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:07.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:07.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:07 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T20:04:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:08 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T20:04:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:08.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 119 op/s 2023-10-01T20:04:08.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:08.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:08.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:08.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:08.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:08.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:09.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.435 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.435 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.435 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.435 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.435 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.436 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.436 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.436 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.436 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.436 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:09.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.439 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:09.439 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:09.439 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:09.439 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:09.439 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T20:04:10.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:10 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T20:04:10.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2023-10-01T20:04:10.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:10.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:10.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:10.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:10.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:10.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:10.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:11.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.356 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.356 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.357 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.357 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.357 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.357 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:11.357 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.357 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.357 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.357 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:11.357 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.357 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.358 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.358 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.358 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.358 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.358 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.358 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.358 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.358 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:11.358 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.358 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:11.358 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:11 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:12.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:12 smithi057 ceph-mon[108286]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:12.354 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:12 smithi057 ceph-mon[114331]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:12.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:12 smithi087 ceph-mon[109386]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:13.330 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:13 smithi057 systemd[1]: Stopping Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:04:13.330 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:13 smithi057 ceph-mon[108286]: Upgrade: Updating alertmanager.a 2023-10-01T20:04:13.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:13 smithi057 ceph-mon[108286]: Deploying daemon alertmanager.a on smithi057 2023-10-01T20:04:13.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:13 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:04:13.331 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:13 smithi057 ceph-mon[114331]: Upgrade: Updating alertmanager.a 2023-10-01T20:04:13.331 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:13 smithi057 ceph-mon[114331]: Deploying daemon alertmanager.a on smithi057 2023-10-01T20:04:13.331 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:13 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:04:13.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:13 smithi087 ceph-mon[109386]: Upgrade: Updating alertmanager.a 2023-10-01T20:04:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:13 smithi087 ceph-mon[109386]: Deploying daemon alertmanager.a on smithi057 2023-10-01T20:04:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:13 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:04:13.671 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[156331]: level=info ts=2023-10-01T20:04:13.329Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2023-10-01T20:04:13.671 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:13 smithi057 bash[188022]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a 2023-10-01T20:04:13.882 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:04:13 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:04:13.505Z caller=notifier.go:532 level=error component=notifier alertmanager=http://172.21.15.57:9093/api/v2/alerts count=3 msg="Error sending alert" err="Post \"http://172.21.15.57:9093/api/v2/alerts\": dial tcp 172.21.15.57:9093: connect: connection refused" 2023-10-01T20:04:14.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:14 smithi057 ceph-mon[108286]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:14.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:14 smithi057 ceph-mon[114331]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:14.172 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:13 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@alertmanager.a.service: Succeeded. 2023-10-01T20:04:14.172 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:13 smithi057 systemd[1]: Stopped Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:04:14.172 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:14 smithi057 systemd[1]: Starting Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:04:14.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:14 smithi087 ceph-mon[109386]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:14.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:04:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:04:14] "GET /metrics HTTP/1.1" 200 37890 "" "Prometheus/2.43.0" 2023-10-01T20:04:14.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[188187]: ts=2023-10-01T20:04:14.473Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2023-10-01T20:04:14.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[188187]: ts=2023-10-01T20:04:14.473Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2023-10-01T20:04:14.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[188187]: ts=2023-10-01T20:04:14.474Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.57 port=9094 2023-10-01T20:04:14.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[188187]: ts=2023-10-01T20:04:14.476Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2023-10-01T20:04:14.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:14 smithi057 bash[188173]: 25df98a755dc7cf836bc7c59780d0acf660c4023043c5a43701b24c7c8f7db0b 2023-10-01T20:04:14.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:14 smithi057 systemd[1]: Started Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:04:14.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[188187]: ts=2023-10-01T20:04:14.502Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2023-10-01T20:04:14.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[188187]: ts=2023-10-01T20:04:14.503Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2023-10-01T20:04:14.923 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[188187]: ts=2023-10-01T20:04:14.506Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2023-10-01T20:04:14.923 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[188187]: ts=2023-10-01T20:04:14.506Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2023-10-01T20:04:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:15.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:15.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T20:04:15.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:15.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:15.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:15.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:15.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:15.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:15.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:15.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:15.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:15.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:15 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:15.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:15.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:15.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T20:04:15.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:15.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:15.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:15.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:15.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:15.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:15.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T20:04:15.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:15.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:15.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:15.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:15.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:15.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:15.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:15.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:15.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:15.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:15.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:15.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:15 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T20:04:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:16 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:16.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T20:04:16.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:16.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:16.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:16.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:16.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.419 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:16.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T20:04:16.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:16.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:16.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:16.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:16.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:16.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:16.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:16.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:16.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:16.671 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[188187]: ts=2023-10-01T20:04:16.476Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000178609s 2023-10-01T20:04:17.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.343 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.343 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.343 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.343 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.343 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.344 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.346 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.346 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.346 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.346 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.346 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.347 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.347 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.347 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.347 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.347 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.347 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.347 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.347 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.347 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.347 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.347 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.347 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.349 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.349 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.349 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.349 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:17 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:17.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:17.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:17.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:17.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:17.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:17.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:17 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:18.327 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:18 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.328 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:18 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.328 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:18 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.328 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:18 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.328 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:18 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.328 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:18 smithi057 ceph-mon[108286]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-10-01T20:04:18.328 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:18 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.329 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:18 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.329 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:18 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.329 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:18 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.329 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:18 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.329 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:18 smithi057 ceph-mon[114331]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-10-01T20:04:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:18 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:18 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:18 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:18 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:18 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:18 smithi087 ceph-mon[109386]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-10-01T20:04:19.226 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:19 smithi057 ceph-mon[114331]: Upgrade: Updating alertmanager.a 2023-10-01T20:04:19.227 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:19 smithi057 ceph-mon[114331]: Deploying daemon alertmanager.a on smithi057 2023-10-01T20:04:19.227 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:19 smithi057 ceph-mon[108286]: Upgrade: Updating alertmanager.a 2023-10-01T20:04:19.227 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:19 smithi057 ceph-mon[108286]: Deploying daemon alertmanager.a on smithi057 2023-10-01T20:04:19.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:19 smithi087 ceph-mon[109386]: Upgrade: Updating alertmanager.a 2023-10-01T20:04:19.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:19 smithi087 ceph-mon[109386]: Deploying daemon alertmanager.a on smithi057 2023-10-01T20:04:20.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:20 smithi087 ceph-mon[109386]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:20.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:20 smithi057 ceph-mon[108286]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:20.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:20 smithi057 ceph-mon[114331]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:21.148 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:21 smithi057 systemd[1]: Stopping Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:04:21.400 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:21 smithi057 ceph-mon[114331]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-10-01T20:04:21.401 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:21 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[188187]: ts=2023-10-01T20:04:21.262Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2023-10-01T20:04:21.401 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:21 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[188187]: ts=2023-10-01T20:04:21.263Z caller=cluster.go:690 level=info component=cluster msg="gossip not settled but continuing anyway" polls=3 elapsed=6.786355546s 2023-10-01T20:04:21.401 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:21 smithi057 ceph-mon[108286]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-10-01T20:04:21.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:21 smithi087 ceph-mon[109386]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-10-01T20:04:21.671 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:21 smithi057 bash[190108]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a 2023-10-01T20:04:22.172 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:21 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@alertmanager.a.service: Succeeded. 2023-10-01T20:04:22.172 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:21 smithi057 systemd[1]: Stopped Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:04:22.172 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:21 smithi057 systemd[1]: Starting Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:04:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:22 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:22 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:22 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:22 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:22 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:22 smithi087 ceph-mon[109386]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 879 B/s rd, 0 op/s 2023-10-01T20:04:22.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:22 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[114331]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 879 B/s rd, 0 op/s 2023-10-01T20:04:22.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[108286]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 879 B/s rd, 0 op/s 2023-10-01T20:04:22.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:22.673 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:04:22.371Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2023-10-01T20:04:22.673 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:04:22.372Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2023-10-01T20:04:22.674 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:04:22.373Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.57 port=9094 2023-10-01T20:04:22.674 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:04:22.374Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2023-10-01T20:04:22.674 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:22 smithi057 bash[190317]: ee46304105ca817a475daabf0584da05b50a2a06b66df2d54a145e97fa8c9bdc 2023-10-01T20:04:22.674 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:22 smithi057 systemd[1]: Started Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:04:22.674 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:04:22.401Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2023-10-01T20:04:22.674 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:04:22.402Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2023-10-01T20:04:22.674 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:04:22.405Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2023-10-01T20:04:22.674 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:04:22.405Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2023-10-01T20:04:23.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-10-01T20:04:23.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 539 B/s rd, 0 op/s 2023-10-01T20:04:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T20:04:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T20:04:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:23.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:23.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-10-01T20:04:23.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 539 B/s rd, 0 op/s 2023-10-01T20:04:23.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:23.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T20:04:23.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T20:04:23.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:23.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:23.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:23.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:23.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:23 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-10-01T20:04:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 539 B/s rd, 0 op/s 2023-10-01T20:04:23.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:23.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T20:04:23.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-10-01T20:04:23.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:23.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:23.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:23.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:23.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:23.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:23.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:23.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:23.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:23 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.421 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:04:24.375Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000810088s 2023-10-01T20:04:24.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2023-10-01T20:04:24.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: Cluster is now healthy 2023-10-01T20:04:24.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:24.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.890 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:04:24] "GET /metrics HTTP/1.1" 200 37890 "" "Prometheus/2.43.0" 2023-10-01T20:04:24.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2023-10-01T20:04:24.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: Cluster is now healthy 2023-10-01T20:04:24.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:24.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.894 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.895 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.895 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.895 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.895 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.895 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.895 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.895 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.895 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2023-10-01T20:04:24.895 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: Cluster is now healthy 2023-10-01T20:04:24.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.897 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.898 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.898 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.898 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.898 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:24.898 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.898 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.898 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.898 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.899 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.899 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:24.899 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.899 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:24.899 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:24.899 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:24.899 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:24.899 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:04:25.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.476 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.476 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.476 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.476 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.476 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.476 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.476 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.476 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.476 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.476 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.476 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.476 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.479 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.479 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.479 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.479 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:04:25.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:04:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:25.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:25.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:25.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:25.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:25.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:25.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:26 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.888 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.888 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.888 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.888 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.888 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.888 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.890 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.892 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.893 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.893 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.893 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.893 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.895 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.895 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.895 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.895 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.895 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.895 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.895 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.895 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.896 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:26.896 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.896 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.896 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.896 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.896 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.896 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.896 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.896 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.896 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.896 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.896 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.897 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:26.897 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.897 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:26.897 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:26.897 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:26.897 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:26.897 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:26 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.482 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.482 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.482 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.483 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:04:27.483 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.483 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.483 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.483 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.483 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.483 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.483 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.483 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.483 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.483 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.484 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.484 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.484 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.485 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.485 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.485 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.486 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:04:27.486 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.486 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.486 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.486 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.486 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.486 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.486 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.486 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.486 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.486 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.486 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.487 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.488 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.488 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.488 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.488 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.488 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:04:27.488 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.488 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.488 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:04:27.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:04:27.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:27 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:04:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:27.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:27.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:27.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:27.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:27.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:27 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.818 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.818 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.818 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.818 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.818 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.818 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.818 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.818 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.818 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.818 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.818 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.819 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.819 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.819 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.819 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.819 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.819 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.819 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.819 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.819 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.820 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.820 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.820 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.820 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.820 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.821 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.822 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.823 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.824 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:28.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:28.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:28.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:28.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:28.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:28.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:28 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.605 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:29.605 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:29.605 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.605 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.605 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.605 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:04:29.605 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:29.605 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:29.606 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.606 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:29.606 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.606 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.606 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.606 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:29.606 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:29.606 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.606 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:29.606 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.606 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.606 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.607 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.607 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:29.607 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.607 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:29.607 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.607 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.607 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.607 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.607 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.607 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.607 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.607 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.608 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.608 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.608 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.608 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.608 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.608 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.608 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.608 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.608 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:29 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:04:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:29.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:29.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:29.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:29.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:29.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:04:29.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:29.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:29.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:29.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:29.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:29.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:29.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:29.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:29.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:29.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:29.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:29.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:30.406 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:04:31.139 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T20:04:31.140 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 2s ago 24m - - 2023-10-01T20:04:31.140 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 0s ago 24m - - 2023-10-01T20:04:31.140 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (9s) 2s ago 21m 17.2M - 0.25.0 c8568f914cd2 ee46304105ca 2023-10-01T20:04:31.140 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (13m) 0s ago 21m - - 8.3.5 dad864ee21e9 ca8ad8b1ccf8 2023-10-01T20:04:31.140 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (57s) 2s ago 20m 52.2M - 3.5 e1d6a67b021e 7ef2818f7c68 2023-10-01T20:04:31.140 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283,8765 running (51s) 0s ago 24m - - 18.0.0-6505-g37d71fd3 bea6d6056140 b5797dd6a1b2 2023-10-01T20:04:31.140 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (10m) 2s ago 25m 522M - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T20:04:31.140 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (25m) 2s ago 25m 206M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T20:04:31.140 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (24m) 0s ago 24m - 2048M 17.2.0 e1d6a67b021e 4734e639b775 2023-10-01T20:04:31.140 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (24m) 2s ago 24m 105M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T20:04:31.140 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (41s) 2s ago 21m 8442k - 1.5.0 0da6a335fe13 175d4990d5fd 2023-10-01T20:04:31.141 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (36s) 0s ago 21m - - 1.5.0 0da6a335fe13 b2605eeb2dd4 2023-10-01T20:04:31.141 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (24m) 2s ago 24m 168M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T20:04:31.141 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (23m) 2s ago 23m 176M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T20:04:31.141 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (23m) 2s ago 23m 127M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T20:04:31.141 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (23m) 2s ago 23m 194M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T20:04:31.141 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (22m) 0s ago 22m - 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T20:04:31.141 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (22m) 0s ago 22m - 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T20:04:31.141 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (22m) 0s ago 22m - 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T20:04:31.141 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (22m) 0s ago 22m - 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T20:04:31.141 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (26s) 0s ago 21m - - 2.43.0 a07b618ecd1d 79b0216ed806 2023-10-01T20:04:31.141 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (20m) 2s ago 20m 99.9M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T20:04:31.142 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (20m) 0s ago 20m - - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T20:04:31.303 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:31 smithi087 ceph-mon[109386]: Upgrade: Updating grafana.a 2023-10-01T20:04:31.303 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:31 smithi087 ceph-mon[109386]: Deploying daemon grafana.a on smithi087 2023-10-01T20:04:31.303 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:31.303 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:31.303 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:31 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:31.389 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:31 smithi057 ceph-mon[108286]: Upgrade: Updating grafana.a 2023-10-01T20:04:31.389 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:31 smithi057 ceph-mon[108286]: Deploying daemon grafana.a on smithi087 2023-10-01T20:04:31.389 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:31.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:31.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:31.390 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:31 smithi057 ceph-mon[114331]: Upgrade: Updating grafana.a 2023-10-01T20:04:31.390 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:31 smithi057 ceph-mon[114331]: Deploying daemon grafana.a on smithi087 2023-10-01T20:04:31.390 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:31.390 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:31.390 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:31.574 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:04:31.574 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2023-10-01T20:04:31.574 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2023-10-01T20:04:31.574 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:04:31.574 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2023-10-01T20:04:31.574 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:04:31.574 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:04:31.574 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2023-10-01T20:04:31.575 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2023-10-01T20:04:31.575 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:04:31.575 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": {}, 2023-10-01T20:04:31.575 INFO:teuthology.orchestra.run.smithi057.stdout: "rgw": { 2023-10-01T20:04:31.575 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T20:04:31.575 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:04:31.575 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2023-10-01T20:04:31.575 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 13, 2023-10-01T20:04:31.575 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:04:31.576 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T20:04:31.576 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:04:31.955 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:04:31.955 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24", 2023-10-01T20:04:31.955 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2023-10-01T20:04:31.955 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2023-10-01T20:04:31.955 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2023-10-01T20:04:31.955 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2023-10-01T20:04:31.956 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-01T20:04:31.956 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "2/2 daemons upgraded", 2023-10-01T20:04:31.956 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading grafana daemons", 2023-10-01T20:04:31.956 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2023-10-01T20:04:31.956 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:04:32.200 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:32 smithi087 ceph-mon[109386]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:32.200 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:32 smithi087 ceph-mon[109386]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:04:32.200 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:32 smithi087 ceph-mon[109386]: from='client.25375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:32.200 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:32 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1764678054' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:32.200 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:31 smithi087 systemd[1]: Stopping Ceph grafana.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:04:32.201 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:32 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[143573]: t=2023-10-01T20:04:32+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2023-10-01T20:04:32.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:32 smithi057 ceph-mon[108286]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:32.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:32 smithi057 ceph-mon[108286]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:04:32.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:32 smithi057 ceph-mon[108286]: from='client.25375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:32.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:32 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1764678054' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:32.380 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:32 smithi057 ceph-mon[114331]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:32.380 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:32 smithi057 ceph-mon[114331]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-10-01T20:04:32.380 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:32 smithi057 ceph-mon[114331]: from='client.25375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:32.380 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:32 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1764678054' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:32.513 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:32 smithi087 bash[174309]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a 2023-10-01T20:04:32.658 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:04:32 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:04:32.378Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003998446s 2023-10-01T20:04:32.883 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:32 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@grafana.a.service: Succeeded. 2023-10-01T20:04:32.883 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:32 smithi087 systemd[1]: Stopped Ceph grafana.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:04:32.883 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:32 smithi087 systemd[1]: Starting Ceph grafana.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:04:33.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-mon[109386]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:33.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-mon[109386]: from='client.25393 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:33.276 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 bash[174462]: e676c0004015e258441c99d6f961d89d2813e0e5d35826e4db7af3e942612410 2023-10-01T20:04:33.276 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 systemd[1]: Started Ceph grafana.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:04:33.276 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.154533534Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2023-10-01T20:04:33.276 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.15472675Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2023-10-01T20:04:33.276 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.154745001Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2023-10-01T20:04:33.276 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.154752007Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2023-10-01T20:04:33.276 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.154758095Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2023-10-01T20:04:33.276 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.154764231Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2023-10-01T20:04:33.276 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.154770094Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2023-10-01T20:04:33.277 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.154775978Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2023-10-01T20:04:33.277 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.154782017Z level=info msg="Path Home" path=/usr/share/grafana 2023-10-01T20:04:33.277 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.154787766Z level=info msg="Path Data" path=/var/lib/grafana 2023-10-01T20:04:33.277 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.154793534Z level=info msg="Path Logs" path=/var/log/grafana 2023-10-01T20:04:33.277 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.154799394Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2023-10-01T20:04:33.277 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.154805275Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2023-10-01T20:04:33.277 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=settings t=2023-10-01T20:04:33.15481104Z level=info msg="App mode production" 2023-10-01T20:04:33.277 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=sqlstore t=2023-10-01T20:04:33.154857228Z level=info msg="Connecting to DB" dbtype=sqlite3 2023-10-01T20:04:33.277 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=sqlstore t=2023-10-01T20:04:33.154872938Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2023-10-01T20:04:33.277 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.166674033Z level=info msg="Starting DB migrations" 2023-10-01T20:04:33.277 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.170386288Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2023-10-01T20:04:33.277 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.182482674Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2023-10-01T20:04:33.278 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.184491901Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2023-10-01T20:04:33.278 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.186885706Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2023-10-01T20:04:33.278 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.189790862Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2023-10-01T20:04:33.278 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.192627161Z level=info msg="Executing migration" id="Add column preferences.json_data" 2023-10-01T20:04:33.278 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.195493923Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2023-10-01T20:04:33.278 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.198458687Z level=info msg="Executing migration" id="Add preferences index org_id" 2023-10-01T20:04:33.278 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.198992725Z level=info msg="Executing migration" id="Add preferences index user_id" 2023-10-01T20:04:33.278 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.201311137Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2023-10-01T20:04:33.278 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.204132424Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2023-10-01T20:04:33.278 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.207061873Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2023-10-01T20:04:33.278 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.207466979Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2023-10-01T20:04:33.278 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.210451833Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2023-10-01T20:04:33.279 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.213163926Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2023-10-01T20:04:33.279 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.215989152Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2023-10-01T20:04:33.279 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.216471582Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2023-10-01T20:04:33.279 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.219250738Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2023-10-01T20:04:33.279 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.221983632Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2023-10-01T20:04:33.279 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.224552288Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2023-10-01T20:04:33.279 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.225629063Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2023-10-01T20:04:33.279 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.228450887Z level=info msg="Executing migration" id="create provenance_type table" 2023-10-01T20:04:33.279 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.231126496Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2023-10-01T20:04:33.279 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.233858075Z level=info msg="Executing migration" id="create alert_image table" 2023-10-01T20:04:33.279 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.234328062Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2023-10-01T20:04:33.279 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.237075182Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2023-10-01T20:04:33.280 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.239646422Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2023-10-01T20:04:33.280 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.243742381Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2023-10-01T20:04:33.280 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.244308556Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2023-10-01T20:04:33.280 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.24440781Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2023-10-01T20:04:33.280 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.246869705Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2023-10-01T20:04:33.280 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.249589276Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2023-10-01T20:04:33.280 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.252072793Z level=info msg="Executing migration" id="increase max description length to 2048" 2023-10-01T20:04:33.280 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.252401664Z level=info msg="Executing migration" id="create secrets table" 2023-10-01T20:04:33.280 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.255100465Z level=info msg="Executing migration" id="rename data_keys name column to id" 2023-10-01T20:04:33.281 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.260283388Z level=info msg="Executing migration" id="add name column into data_keys" 2023-10-01T20:04:33.281 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.26316437Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2023-10-01T20:04:33.281 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.263510609Z level=info msg="Executing migration" id="rename data_keys name column to label" 2023-10-01T20:04:33.281 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.270017006Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2023-10-01T20:04:33.281 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.275525178Z level=info msg="Executing migration" id="add column hidden to role table" 2023-10-01T20:04:33.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:33 smithi057 ceph-mon[108286]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:33.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:33 smithi057 ceph-mon[108286]: from='client.25393 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:33 smithi057 ceph-mon[114331]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:33.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:33 smithi057 ceph-mon[114331]: from='client.25393 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:04:33.633 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.276823845Z level=info msg="Executing migration" id="create query_history table v1" 2023-10-01T20:04:33.633 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.279620198Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2023-10-01T20:04:33.633 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.282474537Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2023-10-01T20:04:33.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.285040025Z level=info msg="Executing migration" id="teams permissions migration" 2023-10-01T20:04:33.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.287619904Z level=info msg="Executing migration" id="dashboard permissions" 2023-10-01T20:04:33.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.290479248Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2023-10-01T20:04:33.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.293064211Z level=info msg="Executing migration" id="drop managed folder create actions" 2023-10-01T20:04:33.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.295735428Z level=info msg="Executing migration" id="alerting notification permissions" 2023-10-01T20:04:33.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.298335063Z level=info msg="Executing migration" id="create query_history_star table v1" 2023-10-01T20:04:33.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.30109096Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2023-10-01T20:04:33.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.303789965Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2023-10-01T20:04:33.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.306469139Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2023-10-01T20:04:33.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.309176555Z level=info msg="Executing migration" id="create correlation table v1" 2023-10-01T20:04:33.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.311876912Z level=info msg="Executing migration" id="add index correlations.uid" 2023-10-01T20:04:33.635 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.314557519Z level=info msg="Executing migration" id="add index correlations.source_uid" 2023-10-01T20:04:33.635 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.317132769Z level=info msg="Executing migration" id="add correlation config column" 2023-10-01T20:04:33.635 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.318199418Z level=info msg="Executing migration" id="create entity_events table" 2023-10-01T20:04:33.635 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.319190956Z level=info msg="Executing migration" id="create dashboard public config v1" 2023-10-01T20:04:33.635 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.319984397Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2023-10-01T20:04:33.635 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.320079186Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2023-10-01T20:04:33.635 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.320997725Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2023-10-01T20:04:33.635 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.321085157Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2023-10-01T20:04:33.635 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.321840903Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2023-10-01T20:04:33.635 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.32262769Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2023-10-01T20:04:33.635 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.323497554Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2023-10-01T20:04:33.635 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.324347716Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2023-10-01T20:04:33.636 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.325181485Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2023-10-01T20:04:33.636 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.32605405Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2023-10-01T20:04:33.636 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.326895003Z level=info msg="Executing migration" id="Drop public config table" 2023-10-01T20:04:33.636 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.32776718Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2023-10-01T20:04:33.636 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.328607701Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2023-10-01T20:04:33.636 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.329494962Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2023-10-01T20:04:33.636 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.330429904Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2023-10-01T20:04:33.636 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.331162752Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2023-10-01T20:04:33.636 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.334232183Z level=info msg="Executing migration" id="add annotations_enabled column" 2023-10-01T20:04:33.636 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.335329656Z level=info msg="Executing migration" id="add time_selection_enabled column" 2023-10-01T20:04:33.636 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.336460856Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2023-10-01T20:04:33.636 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.337354776Z level=info msg="Executing migration" id="add share column" 2023-10-01T20:04:33.637 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.33849446Z level=info msg="Executing migration" id="create default alerting folders" 2023-10-01T20:04:33.637 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.33935354Z level=info msg="Executing migration" id="create file table" 2023-10-01T20:04:33.637 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.340329621Z level=info msg="Executing migration" id="file table idx: path natural pk" 2023-10-01T20:04:33.637 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.341058367Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2023-10-01T20:04:33.637 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.341958464Z level=info msg="Executing migration" id="create file_meta table" 2023-10-01T20:04:33.637 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.342825216Z level=info msg="Executing migration" id="file table idx: path key" 2023-10-01T20:04:33.637 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.343864178Z level=info msg="Executing migration" id="set path collation in file table" 2023-10-01T20:04:33.637 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.344608069Z level=info msg="Executing migration" id="managed permissions migration" 2023-10-01T20:04:33.637 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.345944438Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2023-10-01T20:04:33.637 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.346773313Z level=info msg="Executing migration" id="RBAC action name migrator" 2023-10-01T20:04:33.637 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.347661372Z level=info msg="Executing migration" id="Add UID column to playlist" 2023-10-01T20:04:33.637 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.348836709Z level=info msg="Executing migration" id="Update uid column values in playlist" 2023-10-01T20:04:33.638 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.349735919Z level=info msg="Executing migration" id="Add index for uid in playlist" 2023-10-01T20:04:33.638 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.350598659Z level=info msg="Executing migration" id="update group index for alert rules" 2023-10-01T20:04:33.638 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.351461601Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2023-10-01T20:04:33.638 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.354408935Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2023-10-01T20:04:33.638 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.357311482Z level=info msg="Executing migration" id="add action column to seed_assignment" 2023-10-01T20:04:33.638 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.360302958Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2023-10-01T20:04:33.638 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.36320876Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2023-10-01T20:04:33.638 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.366146357Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2023-10-01T20:04:33.638 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.376515639Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2023-10-01T20:04:33.639 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.379413516Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2023-10-01T20:04:33.639 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.382323764Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2023-10-01T20:04:33.639 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.388610599Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2023-10-01T20:04:33.639 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.39155724Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2023-10-01T20:04:33.639 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.394372274Z level=info msg="Executing migration" id="create folder table" 2023-10-01T20:04:33.639 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.397271994Z level=info msg="Executing migration" id="Add index for parent_uid" 2023-10-01T20:04:33.639 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.400146872Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2023-10-01T20:04:33.639 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.403068361Z level=info msg="Executing migration" id="Update folder title length" 2023-10-01T20:04:33.639 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.405729439Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2023-10-01T20:04:33.639 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=migrator t=2023-10-01T20:04:33.408655449Z level=info msg="migrations completed" performed=105 skipped=376 duration=238.324578ms 2023-10-01T20:04:33.639 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=sqlstore t=2023-10-01T20:04:33.409056083Z level=info msg="Created default organization" 2023-10-01T20:04:33.639 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=plugin.loader t=2023-10-01T20:04:33.478898272Z level=info msg="Plugin registered" pluginID=input 2023-10-01T20:04:33.640 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=plugin.loader t=2023-10-01T20:04:33.507416249Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2023-10-01T20:04:33.640 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=plugin.loader t=2023-10-01T20:04:33.507443289Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2023-10-01T20:04:33.640 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=secrets t=2023-10-01T20:04:33.507482312Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2023-10-01T20:04:33.640 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=query_data t=2023-10-01T20:04:33.509199404Z level=info msg="Query Service initialization" 2023-10-01T20:04:33.640 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:33 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=live.push_http t=2023-10-01T20:04:33.511694799Z level=info msg="Live Push Gateway initialization" 2023-10-01T20:04:34.343 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=infra.usagestats.collector t=2023-10-01T20:04:34.256172817Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2023-10-01T20:04:34.343 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 930 B/s rd, 0 op/s 2023-10-01T20:04:34.343 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.343 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.343 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.343 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:34.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:34.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T20:04:34.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:34.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:34.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:34.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:34.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.345 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:34.345 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:34.345 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.345 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.345 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:34.345 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.345 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:34.345 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:34.345 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 930 B/s rd, 0 op/s 2023-10-01T20:04:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T20:04:34.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:34.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:34.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:34.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:34.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:34.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:34.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:34.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:34.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:34.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 930 B/s rd, 0 op/s 2023-10-01T20:04:34.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:34.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:34.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:34.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:34.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:34.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:34.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:34.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:34.633 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=provisioning.datasources t=2023-10-01T20:04:34.343470797Z level=info msg="deleted correlations based on configuration" ds_name=Dashboard1 2023-10-01T20:04:34.633 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=provisioning.datasources t=2023-10-01T20:04:34.34350516Z level=info msg="deleted datasource based on configuration" name=Dashboard1 2023-10-01T20:04:34.633 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=provisioning.datasources t=2023-10-01T20:04:34.35478122Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2023-10-01T20:04:34.633 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=provisioning.alerting t=2023-10-01T20:04:34.431435418Z level=info msg="starting to provision alerting" 2023-10-01T20:04:34.633 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=provisioning.alerting t=2023-10-01T20:04:34.431464576Z level=info msg="finished to provision alerting" 2023-10-01T20:04:34.633 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=ngalert.state.manager t=2023-10-01T20:04:34.431640154Z level=info msg="Warming state cache for startup" 2023-10-01T20:04:34.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=grafanaStorageLogger t=2023-10-01T20:04:34.431677091Z level=info msg="storage starting" 2023-10-01T20:04:34.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=http.server t=2023-10-01T20:04:34.433525644Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2023-10-01T20:04:34.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=ngalert.state.manager t=2023-10-01T20:04:34.441933671Z level=info msg="State cache has been initialized" states=0 duration=10.264523ms 2023-10-01T20:04:34.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=ngalert.multiorg.alertmanager t=2023-10-01T20:04:34.441994311Z level=info msg="starting MultiOrg Alertmanager" 2023-10-01T20:04:34.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=ticker t=2023-10-01T20:04:34.442008152Z level=info msg=starting first_tick=2023-10-01T20:04:40Z 2023-10-01T20:04:34.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=sqlstore.transactions t=2023-10-01T20:04:34.497147092Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2023-10-01T20:04:34.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=sqlstore.transactions t=2023-10-01T20:04:34.524008933Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2023-10-01T20:04:34.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:04:34 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a[174476]: logger=sqlstore.transactions t=2023-10-01T20:04:34.535216594Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2023-10-01T20:04:35.091 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:04:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:04:34] "GET /metrics HTTP/1.1" 200 37898 "" "Prometheus/2.43.0" 2023-10-01T20:04:35.300 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:35.300 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T20:04:35.300 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:35.300 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:35.300 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:35.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:35.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.302 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.303 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.303 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.303 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.303 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.304 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.304 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.304 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.304 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.304 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.304 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.304 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.304 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.304 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.304 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.304 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.304 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.305 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.305 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.305 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.305 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.305 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.305 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.305 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:35 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:35.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T20:04:35.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:35.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:35.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:35.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:35.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:35.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-10-01T20:04:35.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:35.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:35.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:35.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:35.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:35.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:35.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:35.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:35.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:35.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:35 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.098 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.098 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.098 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.098 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:36.098 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.098 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:36.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:36.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:36.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:36 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:36.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:36.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:36.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:36 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.241 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.241 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.241 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.241 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.241 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.242 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.242 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.242 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.242 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.242 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.243 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.243 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.243 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.243 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.244 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.244 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.244 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.244 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.244 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.245 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.245 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.245 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.245 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.245 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.247 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.249 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.249 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.249 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.249 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:37 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:37.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:37.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:37.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:37.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:37.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:37.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:37 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:38.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:38 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:38.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.432 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:38.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.433 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:38.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:38.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:38.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:38.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:38.434 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:39.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:39 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:39.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:39.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:39.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:39.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:39.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:39.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:39.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:39.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:39 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: Upgrade: Finalizing container_image settings 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2023-10-01T20:04:40.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:04:40.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2023-10-01T20:04:40.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:04:40.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:04:40.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2023-10-01T20:04:40.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:04:40.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:04:40.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2023-10-01T20:04:40.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:04:40.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2023-10-01T20:04:40.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:04:40.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:04:40.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2023-10-01T20:04:40.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:04:40.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2023-10-01T20:04:40.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: Upgrade: Complete! 2023-10-01T20:04:40.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:04:40.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2023-10-01T20:04:40.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:40.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:40.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:40.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:40.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:40.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:40.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:40 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: Upgrade: Finalizing container_image settings 2023-10-01T20:04:40.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:04:40.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2023-10-01T20:04:40.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2023-10-01T20:04:40.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:04:40.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2023-10-01T20:04:40.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:04:40.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:04:40.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:40.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:04:40.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: Upgrade: Finalizing container_image settings 2023-10-01T20:04:40.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:04:40.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2023-10-01T20:04:40.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2023-10-01T20:04:40.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:04:40.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2023-10-01T20:04:40.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:04:40.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:04:40.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2023-10-01T20:04:40.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:04:40.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:04:40.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2023-10-01T20:04:40.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:04:40.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2023-10-01T20:04:40.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:04:40.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:04:40.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2023-10-01T20:04:40.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:04:40.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2023-10-01T20:04:40.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: Upgrade: Complete! 2023-10-01T20:04:40.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:04:40.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2023-10-01T20:04:40.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:04:40.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2023-10-01T20:04:40.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:04:40.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:04:40.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2023-10-01T20:04:40.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:04:40.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2023-10-01T20:04:40.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:04:40.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:04:40.683 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2023-10-01T20:04:40.683 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:04:40.683 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2023-10-01T20:04:40.683 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.683 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.683 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.683 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.683 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.683 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:04:40.684 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: Upgrade: Complete! 2023-10-01T20:04:40.684 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:04:40.684 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2023-10-01T20:04:40.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:40.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:40.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:40.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:40.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:40.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:40.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.685 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:40.685 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:40.685 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:40.685 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.686 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:04:40.686 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:04:40.686 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:04:40.686 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.686 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.686 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:40.686 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:40 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:41.339 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:41 smithi087 ceph-mon[109386]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:41.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:41 smithi057 ceph-mon[108286]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:41.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:41 smithi057 ceph-mon[114331]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:43 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:04:43.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:43.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:43 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:04:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:43 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:04:44.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:44 smithi087 ceph-mon[109386]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:44.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:44 smithi057 ceph-mon[108286]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:44.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:44 smithi057 ceph-mon[114331]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:45.107 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:04:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:04:44] "GET /metrics HTTP/1.1" 200 37895 "" "Prometheus/2.43.0" 2023-10-01T20:04:45.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:45 smithi087 ceph-mon[109386]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:45.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:45 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:45.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:45 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:46.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:46 smithi087 ceph-mon[109386]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:46.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:46 smithi057 ceph-mon[108286]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:46.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:46 smithi057 ceph-mon[114331]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:47.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:47 smithi057 ceph-mon[108286]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:47.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:47 smithi057 ceph-mon[114331]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:47.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:47 smithi087 ceph-mon[109386]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:49.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:49 smithi087 ceph-mon[109386]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:49.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:49 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:49.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:49 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:49.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:49 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:49.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:49 smithi057 ceph-mon[108286]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:49.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:49 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:49.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:49 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:49.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:49 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:49.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:49 smithi057 ceph-mon[114331]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:49.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:49 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:49.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:49 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:49.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:49 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:04:52.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:52 smithi057 ceph-mon[108286]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:52.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:52 smithi057 ceph-mon[114331]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:52.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:52 smithi087 ceph-mon[109386]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:54.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:54 smithi087 ceph-mon[109386]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:54.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:54 smithi057 ceph-mon[108286]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:54.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:54 smithi057 ceph-mon[114331]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:55.057 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:04:54 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:04:54] "GET /metrics HTTP/1.1" 200 37895 "" "Prometheus/2.43.0" 2023-10-01T20:04:55.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:55 smithi087 ceph-mon[109386]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:55.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:55 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:55.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:55 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:04:56.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:56 smithi087 ceph-mon[109386]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:56.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:56 smithi057 ceph-mon[108286]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:56.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:56 smithi057 ceph-mon[114331]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:04:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:58 smithi087 ceph-mon[109386]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:04:58 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:04:58.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:58 smithi057 ceph-mon[108286]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:58.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:04:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:04:58.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:58 smithi057 ceph-mon[114331]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:04:58.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:04:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:05:00.219 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:00 smithi087 ceph-mon[109386]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:00.219 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:00.220 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:00.220 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:00 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:00.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:00 smithi057 ceph-mon[108286]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:00.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:00.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:00.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:00 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:00 smithi057 ceph-mon[114331]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:00.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:00.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:00 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:02.287 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:02 smithi087 ceph-mon[109386]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:02.348 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:02 smithi057 ceph-mon[108286]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:02.348 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:02 smithi057 ceph-mon[114331]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:02.722 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2023-10-01T20:05:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:05:04.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:04 smithi057 ceph-mon[108286]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:04.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:04 smithi057 ceph-mon[108286]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:04.356 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:04 smithi057 ceph-mon[114331]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:04.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:04 smithi057 ceph-mon[114331]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:04.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:04 smithi087 ceph-mon[109386]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:04.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:04 smithi087 ceph-mon[109386]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:04.730 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2023-10-01T20:05:04.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:05:04 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:05:04] "GET /metrics HTTP/1.1" 200 37897 "" "Prometheus/2.43.0" 2023-10-01T20:05:05.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:05 smithi087 ceph-mon[109386]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:05.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:05 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/2429115414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:05.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:05 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:05.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:05 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/2429115414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:05.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:05 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:05.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:05 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2429115414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:06.321 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)" 2023-10-01T20:05:06.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:06 smithi087 ceph-mon[109386]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:06.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:06 smithi057 ceph-mon[108286]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:06.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:06 smithi057 ceph-mon[114331]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:06.729 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2023-10-01T20:05:07.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:07 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/4013074611' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:07.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:07 smithi087 ceph-mon[109386]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:07.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:07 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/4013074611' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:07.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:07 smithi057 ceph-mon[108286]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:07 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/4013074611' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:07.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:07 smithi057 ceph-mon[114331]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:08.428 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:05:08.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:08 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/843967522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:08.739 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:08 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/843967522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:08.795 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2023-10-01T20:05:08.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:08 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/843967522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:09.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:09 smithi057 ceph-mon[108286]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:09.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:09.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:09.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:09 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:09.585 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:09 smithi057 ceph-mon[114331]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:09.586 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:09.586 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:09.586 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:09 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:09 smithi087 ceph-mon[109386]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:09 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:12.338 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:12 smithi057 ceph-mon[114331]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:12.338 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:12 smithi057 ceph-mon[114331]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:12.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:12 smithi057 ceph-mon[108286]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:12.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:12 smithi057 ceph-mon[108286]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:12.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:12 smithi087 ceph-mon[109386]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:12.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:12 smithi087 ceph-mon[109386]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:12.391 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:05:12.914 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch upgrade status' 2023-10-01T20:05:13.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:13 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:05:13.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:13 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:05:13.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:13 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:05:14.263 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:14 smithi057 ceph-mon[108286]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:14.264 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:14 smithi057 ceph-mon[114331]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:14.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:14 smithi087 ceph-mon[109386]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:14.701 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:05:14.701 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": null, 2023-10-01T20:05:14.701 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": false, 2023-10-01T20:05:14.701 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "", 2023-10-01T20:05:14.701 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2023-10-01T20:05:14.701 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": null, 2023-10-01T20:05:14.701 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "", 2023-10-01T20:05:14.701 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2023-10-01T20:05:14.702 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:05:15.037 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:05:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:05:14] "GET /metrics HTTP/1.1" 200 37896 "" "Prometheus/2.43.0" 2023-10-01T20:05:15.090 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph health detail' 2023-10-01T20:05:15.382 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:15 smithi087 ceph-mon[109386]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:15.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:15 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:15.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:15 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:16.335 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:16 smithi057 ceph-mon[108286]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:16.336 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:16 smithi057 ceph-mon[108286]: from='client.25429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:16.336 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:16 smithi057 ceph-mon[114331]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:16.336 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:16 smithi057 ceph-mon[114331]: from='client.25429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:16 smithi087 ceph-mon[109386]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:16.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:16 smithi087 ceph-mon[109386]: from='client.25429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:16.786 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2023-10-01T20:05:17.220 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mon --hosts $(ceph orch ps | grep mgr.x | awk '"'"'{print $2}'"'"')' 2023-10-01T20:05:17.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:17 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/1945217066' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T20:05:17.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:17 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1945217066' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T20:05:17.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:17 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1945217066' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T20:05:18.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:18 smithi057 ceph-mon[108286]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:18.369 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:18 smithi057 ceph-mon[114331]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:18.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:18 smithi087 ceph-mon[109386]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:20.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:20 smithi087 ceph-mon[109386]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:20.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:20 smithi087 ceph-mon[109386]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:20.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:20 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:20.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:20 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:20.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:20 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:20.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:20 smithi057 ceph-mon[108286]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:20.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:20 smithi057 ceph-mon[108286]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:20.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:20 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:20.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:20 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:20.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:20 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:20.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:20 smithi057 ceph-mon[114331]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:20.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:20 smithi057 ceph-mon[114331]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:20.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:20 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:20.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:20 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:20.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:20 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:20.829 INFO:teuthology.orchestra.run.smithi057.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:05:21.260 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2023-10-01T20:05:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:21 smithi087 ceph-mon[109386]: from='client.25447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "daemon_types": "mon", "hosts": "smithi087", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:21 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:21 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:21 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:21 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:21.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:21 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:21.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:21 smithi057 ceph-mon[108286]: from='client.25447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "daemon_types": "mon", "hosts": "smithi087", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:21 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:21 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:21 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:21 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:21.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:21 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:21 smithi057 ceph-mon[114331]: from='client.25447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "daemon_types": "mon", "hosts": "smithi087", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:21 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:21 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:21.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:21 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:21.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:21 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:21.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:21 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:22.363 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:22 smithi057 ceph-mon[108286]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:22.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:22 smithi057 ceph-mon[108286]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:05:22.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:22 smithi057 ceph-mon[108286]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:05:22.364 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:22 smithi057 ceph-mon[114331]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:22.364 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:22 smithi057 ceph-mon[114331]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:05:22.364 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:22 smithi057 ceph-mon[114331]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:05:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:22 smithi087 ceph-mon[109386]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:22 smithi087 ceph-mon[109386]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:05:22.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:22 smithi087 ceph-mon[109386]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:05:23.113 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:05:23.242 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:23 smithi087 ceph-mon[109386]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:23.242 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:23 smithi087 ceph-mon[109386]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:23.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:23 smithi057 ceph-mon[108286]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:23.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:23 smithi057 ceph-mon[108286]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:23.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:23 smithi057 ceph-mon[114331]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:23.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:23 smithi057 ceph-mon[114331]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:23.823 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T20:05:23.823 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 14s ago 25m - - 2023-10-01T20:05:23.823 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 3s ago 25m - - 2023-10-01T20:05:23.823 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (61s) 14s ago 22m 19.4M - 0.25.0 c8568f914cd2 ee46304105ca 2023-10-01T20:05:23.823 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (51s) 3s ago 21m 86.7M - 9.4.7 2c41d148cca3 e676c0004015 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (110s) 14s ago 21m 52.5M - 3.5 e1d6a67b021e 7ef2818f7c68 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283,8765 running (104s) 3s ago 25m 448M - 18.0.0-6505-g37d71fd3 bea6d6056140 b5797dd6a1b2 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (11m) 14s ago 26m 530M - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (26m) 14s ago 26m 213M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (25m) 3s ago 25m 120M 2048M 17.2.0 e1d6a67b021e 4734e639b775 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (25m) 14s ago 25m 107M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (94s) 14s ago 22m 8640k - 1.5.0 0da6a335fe13 175d4990d5fd 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (89s) 3s ago 22m 10.0M - 1.5.0 0da6a335fe13 b2605eeb2dd4 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (24m) 14s ago 24m 170M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (24m) 14s ago 24m 178M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (24m) 14s ago 24m 129M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (24m) 14s ago 24m 197M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T20:05:23.824 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (23m) 3s ago 23m 189M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T20:05:23.825 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (23m) 3s ago 23m 180M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T20:05:23.825 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (23m) 3s ago 23m 144M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T20:05:23.825 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (23m) 3s ago 22m 177M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T20:05:23.825 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (78s) 3s ago 22m 51.4M - 2.43.0 a07b618ecd1d 79b0216ed806 2023-10-01T20:05:23.825 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (21m) 14s ago 21m 100M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T20:05:23.825 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (21m) 3s ago 21m 96.1M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T20:05:24.263 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:05:24.263 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2023-10-01T20:05:24.263 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2023-10-01T20:05:24.263 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:05:24.263 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2023-10-01T20:05:24.263 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:05:24.263 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:05:24.264 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2023-10-01T20:05:24.264 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2023-10-01T20:05:24.264 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:05:24.264 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": {}, 2023-10-01T20:05:24.264 INFO:teuthology.orchestra.run.smithi057.stdout: "rgw": { 2023-10-01T20:05:24.264 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T20:05:24.264 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:05:24.264 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2023-10-01T20:05:24.264 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 13, 2023-10-01T20:05:24.264 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:05:24.265 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T20:05:24.265 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:05:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: Upgrade: Target is version 18.0.0-6505-g37d71fd3 (reef) 2023-10-01T20:05:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24, digests ['quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24'] 2023-10-01T20:05:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:24.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: Upgrade: Setting container_image for all mgr 2023-10-01T20:05:24.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:24.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2023-10-01T20:05:24.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2023-10-01T20:05:24.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: Upgrade: It appears safe to stop mon.b 2023-10-01T20:05:24.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: from='client.25459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:24.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:24.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:24 smithi087 ceph-mon[109386]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:24.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:24.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: Upgrade: Target is version 18.0.0-6505-g37d71fd3 (reef) 2023-10-01T20:05:24.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24, digests ['quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24'] 2023-10-01T20:05:24.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:24.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:24.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all mgr 2023-10-01T20:05:24.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:24.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2023-10-01T20:05:24.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2023-10-01T20:05:24.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: Upgrade: It appears safe to stop mon.b 2023-10-01T20:05:24.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: from='client.25459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:24.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:24.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:24.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:24.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: Upgrade: Target is version 18.0.0-6505-g37d71fd3 (reef) 2023-10-01T20:05:24.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24, digests ['quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24'] 2023-10-01T20:05:24.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:24.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:24.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all mgr 2023-10-01T20:05:24.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:24.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2023-10-01T20:05:24.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2023-10-01T20:05:24.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: Upgrade: It appears safe to stop mon.b 2023-10-01T20:05:24.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: from='client.25459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:24.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:24.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:24.623 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:05:24.623 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24", 2023-10-01T20:05:24.624 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2023-10-01T20:05:24.624 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mon on host(s) smithi087", 2023-10-01T20:05:24.624 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2023-10-01T20:05:24.624 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "0/1 daemons upgraded", 2023-10-01T20:05:24.625 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mon daemons", 2023-10-01T20:05:24.625 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2023-10-01T20:05:24.625 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:05:25.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:05:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:05:24] "GET /metrics HTTP/1.1" 200 37896 "" "Prometheus/2.43.0" 2023-10-01T20:05:25.505 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:25 smithi087 ceph-mon[109386]: Upgrade: Updating mon.b 2023-10-01T20:05:25.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:25.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:05:25.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:05:25.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:25 smithi087 ceph-mon[109386]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:25.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:25 smithi087 ceph-mon[109386]: Deploying daemon mon.b on smithi087 2023-10-01T20:05:25.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:25 smithi087 ceph-mon[109386]: from='client.? 172.21.15.57:0/447816332' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:25.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:25 smithi087 ceph-mon[109386]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:25.506 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:25 smithi087 ceph-mon[109386]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[114331]: Upgrade: Updating mon.b 2023-10-01T20:05:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:05:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:05:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[114331]: Deploying daemon mon.b on smithi087 2023-10-01T20:05:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/447816332' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[114331]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[114331]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[108286]: Upgrade: Updating mon.b 2023-10-01T20:05:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:05:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:05:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[108286]: Deploying daemon mon.b on smithi087 2023-10-01T20:05:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/447816332' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[108286]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:25 smithi057 ceph-mon[108286]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:25.839 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:25 smithi087 systemd[1]: Stopping Ceph mon.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:05:26.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:25 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-b[109336]: 2023-10-01T20:05:25.836+0000 7f6c4384f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:05:26.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:25 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-b[109336]: 2023-10-01T20:05:25.836+0000 7f6c4384f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-01T20:05:26.543 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 bash[178546]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-b 2023-10-01T20:05:26.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.b.service: Succeeded. 2023-10-01T20:05:26.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 systemd[1]: Stopped Ceph mon.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:05:26.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 systemd[1]: Starting Ceph mon.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:05:27.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: set uid:gid to 167:167 (ceph:ceph) 2023-10-01T20:05:27.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev), process ceph-mon, pid 7 2023-10-01T20:05:27.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: pidfile_write: ignore empty --pid-file 2023-10-01T20:05:27.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: load: jerasure load: lrc 2023-10-01T20:05:27.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 bash[178696]: 805778222e17a8e5d4fa629d67c5a769cd39c7f4200e832e81cdd43db05b572e 2023-10-01T20:05:27.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: RocksDB version: 7.9.2 2023-10-01T20:05:27.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Git sha 0 2023-10-01T20:05:27.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Compile date 2023-10-01 17:30:24 2023-10-01T20:05:27.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: DB SUMMARY 2023-10-01T20:05:27.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: DB Session ID: I9LP6B1VPD9TIVQOLWRG 2023-10-01T20:05:27.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: CURRENT file: CURRENT 2023-10-01T20:05:27.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: IDENTITY file: IDENTITY 2023-10-01T20:05:27.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: MANIFEST file: MANIFEST-000009 size: 3612 Bytes 2023-10-01T20:05:27.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 1, files: 000066.sst 2023-10-01T20:05:27.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000064.log size: 626469 ; 2023-10-01T20:05:27.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.error_if_exists: 0 2023-10-01T20:05:27.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.create_if_missing: 0 2023-10-01T20:05:27.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.paranoid_checks: 1 2023-10-01T20:05:27.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-01T20:05:27.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-01T20:05:27.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-01T20:05:27.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.env: 0x562edba79da0 2023-10-01T20:05:27.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.fs: PosixFileSystem 2023-10-01T20:05:27.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.info_log: 0x562edd3b35a0 2023-10-01T20:05:27.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_file_opening_threads: 16 2023-10-01T20:05:27.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.statistics: (nil) 2023-10-01T20:05:27.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.use_fsync: 0 2023-10-01T20:05:27.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_log_file_size: 0 2023-10-01T20:05:27.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-01T20:05:27.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-01T20:05:27.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.keep_log_file_num: 1000 2023-10-01T20:05:27.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.recycle_log_file_num: 0 2023-10-01T20:05:27.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.allow_fallocate: 1 2023-10-01T20:05:27.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.allow_mmap_reads: 0 2023-10-01T20:05:27.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.allow_mmap_writes: 0 2023-10-01T20:05:27.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.use_direct_reads: 0 2023-10-01T20:05:27.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-01T20:05:27.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.create_missing_column_families: 0 2023-10-01T20:05:27.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.db_log_dir: 2023-10-01T20:05:27.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.wal_dir: 2023-10-01T20:05:27.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-01T20:05:27.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-01T20:05:27.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-01T20:05:27.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-01T20:05:27.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-01T20:05:27.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-01T20:05:27.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.advise_random_on_open: 1 2023-10-01T20:05:27.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.db_write_buffer_size: 0 2023-10-01T20:05:27.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.write_buffer_manager: 0x562edd36b360 2023-10-01T20:05:27.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-01T20:05:27.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-01T20:05:27.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-01T20:05:27.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.rate_limiter: (nil) 2023-10-01T20:05:27.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-01T20:05:27.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.wal_recovery_mode: 2 2023-10-01T20:05:27.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.enable_thread_tracking: 0 2023-10-01T20:05:27.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.enable_pipelined_write: 0 2023-10-01T20:05:27.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.unordered_write: 0 2023-10-01T20:05:27.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-01T20:05:27.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-01T20:05:27.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-01T20:05:27.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-01T20:05:27.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.row_cache: None 2023-10-01T20:05:27.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.wal_filter: None 2023-10-01T20:05:27.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-01T20:05:27.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.allow_ingest_behind: 0 2023-10-01T20:05:27.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.two_write_queues: 0 2023-10-01T20:05:27.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.manual_wal_flush: 0 2023-10-01T20:05:27.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.wal_compression: 0 2023-10-01T20:05:27.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.atomic_flush: 0 2023-10-01T20:05:27.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-01T20:05:27.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-01T20:05:27.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-01T20:05:27.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.log_readahead_size: 0 2023-10-01T20:05:27.393 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-01T20:05:27.393 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.best_efforts_recovery: 0 2023-10-01T20:05:27.393 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-01T20:05:27.393 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-01T20:05:27.393 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.allow_data_in_errors: 0 2023-10-01T20:05:27.394 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.db_host_id: __hostname__ 2023-10-01T20:05:27.394 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-01T20:05:27.394 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_background_jobs: 2 2023-10-01T20:05:27.394 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_background_compactions: -1 2023-10-01T20:05:27.395 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_subcompactions: 1 2023-10-01T20:05:27.395 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-01T20:05:27.395 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-01T20:05:27.395 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-01T20:05:27.395 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_total_wal_size: 0 2023-10-01T20:05:27.395 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-01T20:05:27.395 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-01T20:05:27.395 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-01T20:05:27.395 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-01T20:05:27.396 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_open_files: -1 2023-10-01T20:05:27.396 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bytes_per_sync: 0 2023-10-01T20:05:27.396 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-01T20:05:27.396 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-01T20:05:27.396 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_readahead_size: 0 2023-10-01T20:05:27.396 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_background_flushes: -1 2023-10-01T20:05:27.396 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Compression algorithms supported: 2023-10-01T20:05:27.396 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-01T20:05:27.396 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: kZSTD supported: 0 2023-10-01T20:05:27.396 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: kXpressCompression supported: 0 2023-10-01T20:05:27.396 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: kLZ4HCCompression supported: 1 2023-10-01T20:05:27.397 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: kZlibCompression supported: 1 2023-10-01T20:05:27.397 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: kSnappyCompression supported: 1 2023-10-01T20:05:27.397 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: kLZ4Compression supported: 1 2023-10-01T20:05:27.397 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: kBZip2Compression supported: 0 2023-10-01T20:05:27.397 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-01T20:05:27.398 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-01T20:05:27.398 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000009 2023-10-01T20:05:27.398 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-01T20:05:27.398 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-01T20:05:27.398 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.merge_operator: 2023-10-01T20:05:27.398 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_filter: None 2023-10-01T20:05:27.399 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_filter_factory: None 2023-10-01T20:05:27.399 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.sst_partitioner_factory: None 2023-10-01T20:05:27.399 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-01T20:05:27.399 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-01T20:05:27.399 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562edd3b36a0) 2023-10-01T20:05:27.399 INFO:journalctl@ceph.mon.b.smithi087.stdout: cache_index_and_filter_blocks: 1 2023-10-01T20:05:27.399 INFO:journalctl@ceph.mon.b.smithi087.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-01T20:05:27.399 INFO:journalctl@ceph.mon.b.smithi087.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-01T20:05:27.399 INFO:journalctl@ceph.mon.b.smithi087.stdout: pin_top_level_index_and_filter: 1 2023-10-01T20:05:27.400 INFO:journalctl@ceph.mon.b.smithi087.stdout: index_type: 0 2023-10-01T20:05:27.400 INFO:journalctl@ceph.mon.b.smithi087.stdout: data_block_index_type: 0 2023-10-01T20:05:27.400 INFO:journalctl@ceph.mon.b.smithi087.stdout: index_shortening: 1 2023-10-01T20:05:27.400 INFO:journalctl@ceph.mon.b.smithi087.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-01T20:05:27.400 INFO:journalctl@ceph.mon.b.smithi087.stdout: checksum: 4 2023-10-01T20:05:27.400 INFO:journalctl@ceph.mon.b.smithi087.stdout: no_block_cache: 0 2023-10-01T20:05:27.400 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_cache: 0x562edd4091f0 2023-10-01T20:05:27.400 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_cache_name: BinnedLRUCache 2023-10-01T20:05:27.400 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_cache_options: 2023-10-01T20:05:27.401 INFO:journalctl@ceph.mon.b.smithi087.stdout: capacity : 536870912 2023-10-01T20:05:27.402 INFO:journalctl@ceph.mon.b.smithi087.stdout: num_shard_bits : 4 2023-10-01T20:05:27.402 INFO:journalctl@ceph.mon.b.smithi087.stdout: strict_capacity_limit : 0 2023-10-01T20:05:27.402 INFO:journalctl@ceph.mon.b.smithi087.stdout: high_pri_pool_ratio: 0.000 2023-10-01T20:05:27.402 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_cache_compressed: (nil) 2023-10-01T20:05:27.402 INFO:journalctl@ceph.mon.b.smithi087.stdout: persistent_cache: (nil) 2023-10-01T20:05:27.402 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_size: 4096 2023-10-01T20:05:27.402 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_size_deviation: 10 2023-10-01T20:05:27.402 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_restart_interval: 16 2023-10-01T20:05:27.402 INFO:journalctl@ceph.mon.b.smithi087.stdout: index_block_restart_interval: 1 2023-10-01T20:05:27.402 INFO:journalctl@ceph.mon.b.smithi087.stdout: metadata_block_size: 4096 2023-10-01T20:05:27.403 INFO:journalctl@ceph.mon.b.smithi087.stdout: partition_filters: 0 2023-10-01T20:05:27.403 INFO:journalctl@ceph.mon.b.smithi087.stdout: use_delta_encoding: 1 2023-10-01T20:05:27.403 INFO:journalctl@ceph.mon.b.smithi087.stdout: filter_policy: bloomfilter 2023-10-01T20:05:27.403 INFO:journalctl@ceph.mon.b.smithi087.stdout: whole_key_filtering: 1 2023-10-01T20:05:27.403 INFO:journalctl@ceph.mon.b.smithi087.stdout: verify_compression: 0 2023-10-01T20:05:27.403 INFO:journalctl@ceph.mon.b.smithi087.stdout: read_amp_bytes_per_bit: 0 2023-10-01T20:05:27.403 INFO:journalctl@ceph.mon.b.smithi087.stdout: format_version: 5 2023-10-01T20:05:27.403 INFO:journalctl@ceph.mon.b.smithi087.stdout: enable_index_compression: 1 2023-10-01T20:05:27.403 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_align: 0 2023-10-01T20:05:27.403 INFO:journalctl@ceph.mon.b.smithi087.stdout: max_auto_readahead_size: 262144 2023-10-01T20:05:27.403 INFO:journalctl@ceph.mon.b.smithi087.stdout: prepopulate_block_cache: 0 2023-10-01T20:05:27.404 INFO:journalctl@ceph.mon.b.smithi087.stdout: initial_auto_readahead_size: 8192 2023-10-01T20:05:27.404 INFO:journalctl@ceph.mon.b.smithi087.stdout: num_file_reads_for_auto_readahead: 2 2023-10-01T20:05:27.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.write_buffer_size: 33554432 2023-10-01T20:05:27.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_write_buffer_number: 2 2023-10-01T20:05:27.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compression: NoCompression 2023-10-01T20:05:27.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bottommost_compression: Disabled 2023-10-01T20:05:27.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.prefix_extractor: nullptr 2023-10-01T20:05:27.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-01T20:05:27.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.num_levels: 7 2023-10-01T20:05:27.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-01T20:05:27.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-01T20:05:27.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-01T20:05:27.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-01T20:05:27.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-01T20:05:27.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-01T20:05:27.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-01T20:05:27.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-01T20:05:27.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-01T20:05:27.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-01T20:05:27.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-01T20:05:27.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-01T20:05:27.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-01T20:05:27.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compression_opts.level: 32767 2023-10-01T20:05:27.406 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compression_opts.strategy: 0 2023-10-01T20:05:27.406 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-01T20:05:27.406 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-01T20:05:27.406 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-01T20:05:27.406 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-01T20:05:27.406 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compression_opts.enabled: false 2023-10-01T20:05:27.406 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-01T20:05:27.406 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-01T20:05:27.406 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-01T20:05:27.406 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-01T20:05:27.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.target_file_size_base: 67108864 2023-10-01T20:05:27.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-01T20:05:27.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-01T20:05:27.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-01T20:05:27.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-01T20:05:27.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-01T20:05:27.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-01T20:05:27.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-01T20:05:27.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-01T20:05:27.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-01T20:05:27.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-01T20:05:27.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-01T20:05:27.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-01T20:05:27.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-01T20:05:27.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-01T20:05:27.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.arena_block_size: 1048576 2023-10-01T20:05:27.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-01T20:05:27.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-01T20:05:27.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.disable_auto_compactions: 0 2023-10-01T20:05:27.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-01T20:05:27.409 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-01T20:05:27.409 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-01T20:05:27.409 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-01T20:05:27.409 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-01T20:05:27.409 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-01T20:05:27.409 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-01T20:05:27.409 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-01T20:05:27.409 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-01T20:05:27.409 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-01T20:05:27.409 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.table_properties_collectors: 2023-10-01T20:05:27.409 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.inplace_update_support: 0 2023-10-01T20:05:27.409 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-01T20:05:27.410 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-01T20:05:27.410 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-01T20:05:27.410 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-01T20:05:27.410 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.bloom_locality: 0 2023-10-01T20:05:27.410 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.max_successive_merges: 0 2023-10-01T20:05:27.410 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-01T20:05:27.410 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.paranoid_file_checks: 0 2023-10-01T20:05:27.410 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.force_consistency_checks: 1 2023-10-01T20:05:27.410 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.report_bg_io_stats: 0 2023-10-01T20:05:27.410 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.ttl: 2592000 2023-10-01T20:05:27.410 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-01T20:05:27.410 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-01T20:05:27.411 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-01T20:05:27.411 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.enable_blob_files: false 2023-10-01T20:05:27.411 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.min_blob_size: 0 2023-10-01T20:05:27.411 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.blob_file_size: 268435456 2023-10-01T20:05:27.411 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-01T20:05:27.411 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-01T20:05:27.411 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-01T20:05:27.411 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-01T20:05:27.411 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-01T20:05:27.411 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.blob_file_starting_level: 0 2023-10-01T20:05:27.411 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-01T20:05:27.411 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 66.sst 2023-10-01T20:05:27.412 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2023-10-01T20:05:27.412 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 68, last_sequence is 36627, log_number is 64,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-01T20:05:27.412 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 64 2023-10-01T20:05:27.412 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ac313c2a-1aff-4c07-87af-2970e086f073 2023-10-01T20:05:27.412 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696190726958848, "job": 1, "event": "recovery_started", "wal_files": [64]} 2023-10-01T20:05:27.412 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #64 mode 2 2023-10-01T20:05:27.412 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 systemd[1]: Started Ceph mon.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:05:27.412 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696190726961172, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 69, "file_size": 412601, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 36622, "largest_seqno": 37034, "table_properties": {"data_size": 410053, "index_size": 1086, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 453, "raw_key_size": 4143, "raw_average_key_size": 24, "raw_value_size": 406249, "raw_average_value_size": 2389, "num_data_blocks": 47, "num_entries": 170, "num_filter_entries": 170, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696190726, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ac313c2a-1aff-4c07-87af-2970e086f073", "db_session_id": "I9LP6B1VPD9TIVQOLWRG", "orig_file_number": 69, "seqno_to_time_mapping": "N/A"}} 2023-10-01T20:05:27.412 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696190726961248, "job": 1, "event": "recovery_finished"} 2023-10-01T20:05:27.412 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [db/version_set.cc:5047] Creating manifest 71 2023-10-01T20:05:27.412 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2023-10-01T20:05:27.412 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000064.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-01T20:05:27.413 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562edd4a6000 2023-10-01T20:05:27.413 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: DB pointer 0x562edd492000 2023-10-01T20:05:27.413 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-01T20:05:27.413 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-01T20:05:27.413 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** DB Stats ** 2023-10-01T20:05:27.413 INFO:journalctl@ceph.mon.b.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T20:05:27.413 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-01T20:05:27.413 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-01T20:05:27.413 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T20:05:27.413 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-01T20:05:27.413 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-01T20:05:27.413 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T20:05:27.414 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T20:05:27.414 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** Compaction Stats [default] ** 2023-10-01T20:05:27.414 INFO:journalctl@ceph.mon.b.smithi087.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-01T20:05:27.414 INFO:journalctl@ceph.mon.b.smithi087.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-01T20:05:27.414 INFO:journalctl@ceph.mon.b.smithi087.stdout: L0 1/0 402.93 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 326.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-01T20:05:27.414 INFO:journalctl@ceph.mon.b.smithi087.stdout: L6 1/0 8.39 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2023-10-01T20:05:27.414 INFO:journalctl@ceph.mon.b.smithi087.stdout: Sum 2/0 8.78 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 326.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-01T20:05:27.414 INFO:journalctl@ceph.mon.b.smithi087.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 326.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-01T20:05:27.414 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T20:05:27.414 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** Compaction Stats [default] ** 2023-10-01T20:05:27.414 INFO:journalctl@ceph.mon.b.smithi087.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-01T20:05:27.414 INFO:journalctl@ceph.mon.b.smithi087.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T20:05:27.415 INFO:journalctl@ceph.mon.b.smithi087.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 326.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-01T20:05:27.415 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T20:05:27.415 INFO:journalctl@ceph.mon.b.smithi087.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-01T20:05:27.415 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T20:05:27.415 INFO:journalctl@ceph.mon.b.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T20:05:27.415 INFO:journalctl@ceph.mon.b.smithi087.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-01T20:05:27.415 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-01T20:05:27.415 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-01T20:05:27.415 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-01T20:05:27.415 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-01T20:05:27.415 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative compaction: 0.00 GB write, 85.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T20:05:27.416 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval compaction: 0.00 GB write, 85.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T20:05:27.416 INFO:journalctl@ceph.mon.b.smithi087.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-01T20:05:27.416 INFO:journalctl@ceph.mon.b.smithi087.stdout: Block cache BinnedLRUCache@0x562edd4091f0#7 capacity: 512.00 MB usage: 1.70 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-10-01T20:05:27.416 INFO:journalctl@ceph.mon.b.smithi087.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.50 KB,9.53674e-05%) IndexBlock(1,1.20 KB,0.000229478%) Misc(1,0.00 KB,0%) 2023-10-01T20:05:27.416 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T20:05:27.416 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-01T20:05:27.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: starting mon.b rank 2 at public addrs [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] at bind addrs [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T20:05:27.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: mon.b@-1(???) e3 preinit fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T20:05:27.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: mon.b@-1(???).mds e1 new map 2023-10-01T20:05:27.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: mon.b@-1(???).mds e1 print_map 2023-10-01T20:05:27.416 INFO:journalctl@ceph.mon.b.smithi087.stdout: e1 2023-10-01T20:05:27.416 INFO:journalctl@ceph.mon.b.smithi087.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-01T20:05:27.417 INFO:journalctl@ceph.mon.b.smithi087.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-01T20:05:27.417 INFO:journalctl@ceph.mon.b.smithi087.stdout: legacy client fscid: -1 2023-10-01T20:05:27.417 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-10-01T20:05:27.417 INFO:journalctl@ceph.mon.b.smithi087.stdout: No filesystems configured 2023-10-01T20:05:27.417 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: mon.b@-1(???).osd e107 crush map has features 3314933000854323200, adjusting msgr requires 2023-10-01T20:05:27.417 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: mon.b@-1(???).osd e107 crush map has features 432629239337189376, adjusting msgr requires 2023-10-01T20:05:27.417 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: mon.b@-1(???).osd e107 crush map has features 432629239337189376, adjusting msgr requires 2023-10-01T20:05:27.417 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: mon.b@-1(???).osd e107 crush map has features 432629239337189376, adjusting msgr requires 2023-10-01T20:05:27.417 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:26 smithi087 ceph-mon[178735]: mon.b@-1(???).paxosservice(auth 1..26) refresh upgraded, format 0 -> 3 2023-10-01T20:05:28.274 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: mon.b calling monitor election 2023-10-01T20:05:28.274 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: mon.c calling monitor election 2023-10-01T20:05:28.274 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: mon.a calling monitor election 2023-10-01T20:05:28.274 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T20:05:28.274 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} 2023-10-01T20:05:28.274 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: fsmap 2023-10-01T20:05:28.274 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: osdmap e107: 8 total, 8 up, 8 in 2023-10-01T20:05:28.274 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: mgrmap e43: y(active, since 2m), standbys: x 2023-10-01T20:05:28.274 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: overall HEALTH_OK 2023-10-01T20:05:28.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.275 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.276 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.277 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.277 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.277 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.277 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.277 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.277 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.277 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.277 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.277 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.277 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.277 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.277 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:28 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: mon.b calling monitor election 2023-10-01T20:05:28.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: mon.c calling monitor election 2023-10-01T20:05:28.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: mon.a calling monitor election 2023-10-01T20:05:28.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T20:05:28.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} 2023-10-01T20:05:28.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: fsmap 2023-10-01T20:05:28.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: osdmap e107: 8 total, 8 up, 8 in 2023-10-01T20:05:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: mgrmap e43: y(active, since 2m), standbys: x 2023-10-01T20:05:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: overall HEALTH_OK 2023-10-01T20:05:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: mon.b calling monitor election 2023-10-01T20:05:28.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: mon.c calling monitor election 2023-10-01T20:05:28.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: mon.a calling monitor election 2023-10-01T20:05:28.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T20:05:28.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} 2023-10-01T20:05:28.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: fsmap 2023-10-01T20:05:28.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: osdmap e107: 8 total, 8 up, 8 in 2023-10-01T20:05:28.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: mgrmap e43: y(active, since 2m), standbys: x 2023-10-01T20:05:28.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: overall HEALTH_OK 2023-10-01T20:05:28.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:28.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:28.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:28.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:28.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:28.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:28.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:28 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:29.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:29 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:29.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.661 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.661 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.661 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.661 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.661 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.661 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:29.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.664 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.665 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.665 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.665 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.665 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.665 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.665 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.665 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.665 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.665 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.665 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.665 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.665 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.666 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:29.666 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:29.666 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.666 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:29.666 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:29.666 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:29.666 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:29.666 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:29 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:30 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:30.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:30.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:30.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:30.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:30.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:30.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:30 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:31.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:31 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:31.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:31.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:31.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:31.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:31.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:31.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:31.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:32 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:32.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:32.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:32.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:32.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:32.431 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:32 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:32.431 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:05:31 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:05:31.970+0000 7f4c916e3700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-01T20:05:33.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:33.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:33.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:33.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all crash 2023-10-01T20:05:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all mds 2023-10-01T20:05:33.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all rbd-mirror 2023-10-01T20:05:33.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all cephfs-mirror 2023-10-01T20:05:33.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2023-10-01T20:05:33.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all ceph-exporter 2023-10-01T20:05:33.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all nfs 2023-10-01T20:05:33.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all nvmeof 2023-10-01T20:05:33.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all node-exporter 2023-10-01T20:05:33.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all prometheus 2023-10-01T20:05:33.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all alertmanager 2023-10-01T20:05:33.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all grafana 2023-10-01T20:05:33.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all loki 2023-10-01T20:05:33.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all promtail 2023-10-01T20:05:33.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Finalizing container_image settings 2023-10-01T20:05:33.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:05:33.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2023-10-01T20:05:33.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2023-10-01T20:05:33.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:05:33.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2023-10-01T20:05:33.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:05:33.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:05:33.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:33.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:33.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:33.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:33.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:33.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all crash 2023-10-01T20:05:33.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all mds 2023-10-01T20:05:33.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all rbd-mirror 2023-10-01T20:05:33.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all cephfs-mirror 2023-10-01T20:05:33.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2023-10-01T20:05:33.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all ceph-exporter 2023-10-01T20:05:33.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all nfs 2023-10-01T20:05:33.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all nvmeof 2023-10-01T20:05:33.430 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2023-10-01T20:05:33.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:05:33.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:05:33.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2023-10-01T20:05:33.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:05:33.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2023-10-01T20:05:33.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:05:33.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:05:33.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2023-10-01T20:05:33.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:05:33.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2023-10-01T20:05:33.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: Upgrade: Complete! 2023-10-01T20:05:33.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:05:33.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2023-10-01T20:05:33.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:33.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:33.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:33.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:33.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:33.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:33.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:33.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.437 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all node-exporter 2023-10-01T20:05:33.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all prometheus 2023-10-01T20:05:33.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all alertmanager 2023-10-01T20:05:33.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all grafana 2023-10-01T20:05:33.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all loki 2023-10-01T20:05:33.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all promtail 2023-10-01T20:05:33.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Finalizing container_image settings 2023-10-01T20:05:33.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:05:33.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2023-10-01T20:05:33.439 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2023-10-01T20:05:33.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:05:33.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2023-10-01T20:05:33.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:05:33.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:05:33.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2023-10-01T20:05:33.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:05:33.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:05:33.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2023-10-01T20:05:33.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.440 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:05:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2023-10-01T20:05:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:05:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:05:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2023-10-01T20:05:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:05:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2023-10-01T20:05:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: Upgrade: Complete! 2023-10-01T20:05:33.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:05:33.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2023-10-01T20:05:33.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:33.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:33.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:33.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:33.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:33.442 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:33.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:33.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.443 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:33 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:33.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:33.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:33.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:05:33.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:33.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all crash 2023-10-01T20:05:33.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all mds 2023-10-01T20:05:33.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all rbd-mirror 2023-10-01T20:05:33.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all cephfs-mirror 2023-10-01T20:05:33.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2023-10-01T20:05:33.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all ceph-exporter 2023-10-01T20:05:33.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all nfs 2023-10-01T20:05:33.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all nvmeof 2023-10-01T20:05:33.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all node-exporter 2023-10-01T20:05:33.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all prometheus 2023-10-01T20:05:33.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all alertmanager 2023-10-01T20:05:33.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all grafana 2023-10-01T20:05:33.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all loki 2023-10-01T20:05:33.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:05:33.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all promtail 2023-10-01T20:05:33.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2023-10-01T20:05:33.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Finalizing container_image settings 2023-10-01T20:05:33.639 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:05:33.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2023-10-01T20:05:33.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2023-10-01T20:05:33.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:05:33.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2023-10-01T20:05:33.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:05:33.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:05:33.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2023-10-01T20:05:33.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:05:33.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:05:33.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2023-10-01T20:05:33.640 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:05:33.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2023-10-01T20:05:33.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:05:33.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:05:33.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2023-10-01T20:05:33.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:05:33.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2023-10-01T20:05:33.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.641 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:05:33.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: Upgrade: Complete! 2023-10-01T20:05:33.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:05:33.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2023-10-01T20:05:33.642 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:33.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:33.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:33.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:33.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:05:33.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:05:33.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:05:33.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:33.643 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:33 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:34.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:34 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:34.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:34 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:34.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:05:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:05:34] "GET /metrics HTTP/1.1" 200 37828 "" "Prometheus/2.43.0" 2023-10-01T20:05:34.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:34 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:36.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:36 smithi087 ceph-mon[178735]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:36.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:36 smithi057 ceph-mon[108286]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:36.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:36 smithi057 ceph-mon[114331]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:38.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:38 smithi087 ceph-mon[178735]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:38.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:38 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:38.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:38 smithi057 ceph-mon[108286]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:38.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:38 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:38.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:38 smithi057 ceph-mon[114331]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:38.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:38 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:40.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:40 smithi087 ceph-mon[178735]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:40.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:40 smithi057 ceph-mon[108286]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:40.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:40 smithi057 ceph-mon[114331]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:42 smithi087 ceph-mon[178735]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:42 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:05:42.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:42 smithi057 ceph-mon[114331]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:42.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:42 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:05:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:42 smithi057 ceph-mon[108286]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:42 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:05:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:43 smithi087 ceph-mon[178735]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:43.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:43 smithi057 ceph-mon[108286]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:43.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:43 smithi057 ceph-mon[114331]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:44.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:44 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:44.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:44 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:44.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:44 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:45.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:05:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:05:44] "GET /metrics HTTP/1.1" 200 37916 "" "Prometheus/2.43.0" 2023-10-01T20:05:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:46 smithi087 ceph-mon[178735]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:46.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:46 smithi057 ceph-mon[108286]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:46.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:46 smithi057 ceph-mon[114331]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:48 smithi087 ceph-mon[178735]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:48.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:48 smithi057 ceph-mon[108286]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:48.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:48 smithi057 ceph-mon[114331]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:50.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:50 smithi087 ceph-mon[178735]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:50.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:50 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:50.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:50 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:50.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:50 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:50.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:50 smithi057 ceph-mon[108286]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:50.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:50 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:50.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:50 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:50.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:50 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:50 smithi057 ceph-mon[114331]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:50 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:50.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:50 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:50.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:50 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:52.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:52 smithi087 ceph-mon[178735]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:52.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:52 smithi057 ceph-mon[108286]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:52.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:52 smithi057 ceph-mon[114331]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:54.322 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:53 smithi087 ceph-mon[178735]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:54.323 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:53 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:54.323 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:53 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:54.323 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:53 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:54.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:53 smithi057 ceph-mon[108286]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:54.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:53 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:54.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:53 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:54.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:53 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:54.361 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:53 smithi057 ceph-mon[114331]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:54.361 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:53 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:54.361 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:53 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:54.361 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:53 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:05:54.991 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:05:54 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:05:54] "GET /metrics HTTP/1.1" 200 37916 "" "Prometheus/2.43.0" 2023-10-01T20:05:55.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:54 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:55.311 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:54 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:55.311 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:54 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:05:55.524 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch ps' 2023-10-01T20:05:56.310 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:56 smithi057 ceph-mon[114331]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:56.310 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:56 smithi057 ceph-mon[114331]: from='client.25483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:56.310 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:56 smithi057 ceph-mon[108286]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:56.310 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:56 smithi057 ceph-mon[108286]: from='client.25483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:56.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:56 smithi087 ceph-mon[178735]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:05:56.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:56 smithi087 ceph-mon[178735]: from='client.25483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:57.256 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T20:05:57.256 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 7s ago 26m - - 2023-10-01T20:05:57.256 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 4s ago 26m - - 2023-10-01T20:05:57.257 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (95s) 7s ago 22m 19.4M - 0.25.0 c8568f914cd2 ee46304105ca 2023-10-01T20:05:57.257 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (84s) 4s ago 22m 86.9M - 9.4.7 2c41d148cca3 e676c0004015 2023-10-01T20:05:57.257 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (2m) 7s ago 21m 54.8M - 3.5 e1d6a67b021e 7ef2818f7c68 2023-10-01T20:05:57.257 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283,8765 running (2m) 4s ago 25m 449M - 18.0.0-6505-g37d71fd3 bea6d6056140 b5797dd6a1b2 2023-10-01T20:05:57.257 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (12m) 7s ago 27m 537M - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T20:05:57.257 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (27m) 7s ago 27m 220M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T20:05:57.257 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (30s) 4s ago 25m 35.3M 2048M 18.0.0-6505-g37d71fd3 bea6d6056140 805778222e17 2023-10-01T20:05:57.257 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (26m) 7s ago 26m 112M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T20:05:57.257 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (2m) 7s ago 22m 8778k - 1.5.0 0da6a335fe13 175d4990d5fd 2023-10-01T20:05:57.258 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (2m) 4s ago 22m 18.2M - 1.5.0 0da6a335fe13 b2605eeb2dd4 2023-10-01T20:05:57.258 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (25m) 7s ago 25m 173M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T20:05:57.258 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (25m) 7s ago 25m 181M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T20:05:57.258 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (24m) 7s ago 24m 131M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T20:05:57.258 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (24m) 7s ago 24m 200M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T20:05:57.258 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (24m) 4s ago 24m 193M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T20:05:57.258 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (24m) 4s ago 24m 184M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T20:05:57.258 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (23m) 4s ago 23m 147M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T20:05:57.258 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (23m) 4s ago 23m 181M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T20:05:57.258 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (112s) 4s ago 23m 53.1M - 2.43.0 a07b618ecd1d 79b0216ed806 2023-10-01T20:05:57.258 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (22m) 7s ago 22m 100M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T20:05:57.258 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (22m) 4s ago 22m 96.8M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T20:05:57.678 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions | jq -e '"'"'.mon | length == 2'"'"'' 2023-10-01T20:05:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:58 smithi087 ceph-mon[178735]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:58.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:58 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:05:58.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:58 smithi057 ceph-mon[108286]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:58.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:58 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:05:58.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:58 smithi057 ceph-mon[114331]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:05:58.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:58 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:05:59.300 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:05:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:05:59 smithi087 ceph-mon[178735]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:59.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:05:59 smithi057 ceph-mon[108286]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:59.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:05:59 smithi057 ceph-mon[114331]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:05:59.689 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch upgrade status' 2023-10-01T20:06:00.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:00 smithi087 ceph-mon[178735]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:00.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:00 smithi087 ceph-mon[178735]: from='client.? 172.21.15.57:0/1761173698' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:00.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:00 smithi057 ceph-mon[108286]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:00.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:00 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/1761173698' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:00 smithi057 ceph-mon[114331]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:00.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:00 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/1761173698' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:01.238 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:06:01.238 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": null, 2023-10-01T20:06:01.238 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": false, 2023-10-01T20:06:01.238 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "", 2023-10-01T20:06:01.238 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2023-10-01T20:06:01.239 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": null, 2023-10-01T20:06:01.239 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "", 2023-10-01T20:06:01.239 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2023-10-01T20:06:01.239 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:06:01.730 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph health detail' 2023-10-01T20:06:02.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:02 smithi087 ceph-mon[178735]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:02.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:02 smithi057 ceph-mon[108286]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:02.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:02 smithi057 ceph-mon[114331]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:03.343 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2023-10-01T20:06:03.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:03 smithi057 ceph-mon[108286]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:03.379 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:03 smithi057 ceph-mon[114331]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:03.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:03 smithi087 ceph-mon[178735]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:03.751 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mon --hosts $(ceph orch ps | grep mgr.y | awk '"'"'{print $2}'"'"')' 2023-10-01T20:06:04.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:04 smithi087 ceph-mon[178735]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:04.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:04 smithi087 ceph-mon[178735]: from='client.? 172.21.15.57:0/3542780887' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T20:06:04.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:04 smithi057 ceph-mon[108286]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:04.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:04 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/3542780887' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T20:06:04.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:04 smithi057 ceph-mon[114331]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:04.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:04 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/3542780887' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T20:06:04.912 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:04 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:06:04] "GET /metrics HTTP/1.1" 200 37912 "" "Prometheus/2.43.0" 2023-10-01T20:06:05.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:05 smithi057 ceph-mon[108286]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:05.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:05 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:05.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:05 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:06.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:06 smithi087 ceph-mon[178735]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:06.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:06 smithi087 ceph-mon[178735]: from='client.15627 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:06.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:06 smithi087 ceph-mon[178735]: from='client.25519 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "daemon_types": "mon", "hosts": "smithi057", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:06.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:06 smithi057 ceph-mon[108286]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:06.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:06 smithi057 ceph-mon[108286]: from='client.15627 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:06.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:06 smithi057 ceph-mon[108286]: from='client.25519 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "daemon_types": "mon", "hosts": "smithi057", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:06.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:06 smithi057 ceph-mon[114331]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:06.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:06 smithi057 ceph-mon[114331]: from='client.15627 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:06.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:06 smithi057 ceph-mon[114331]: from='client.25519 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "daemon_types": "mon", "hosts": "smithi057", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:07.138 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:07 smithi057 ceph-mon[108286]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:07.184 INFO:teuthology.orchestra.run.smithi057.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:06:07.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:07 smithi057 ceph-mon[114331]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:07.594 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2023-10-01T20:06:07.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:07 smithi087 ceph-mon[178735]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:08.308 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[108286]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:06:08.308 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:08.308 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:08.308 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:08.308 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:08.308 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:08.308 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[108286]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:06:08.309 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[114331]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:06:08.309 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:08.309 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:08.309 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:08.309 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:08.309 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:08.309 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:08 smithi057 ceph-mon[114331]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:06:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:08 smithi087 ceph-mon[178735]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:06:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:08 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:08 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:08 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:08 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:08 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:08.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:08 smithi087 ceph-mon[178735]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:06:09.273 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:06:09.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:09 smithi057 ceph-mon[108286]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:09.364 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:09 smithi057 ceph-mon[114331]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:09.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:09 smithi087 ceph-mon[178735]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:10.051 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T20:06:10.051 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 0s ago 26m - - 2023-10-01T20:06:10.051 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 17s ago 26m - - 2023-10-01T20:06:10.051 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (108s) 0s ago 22m 19.5M - 0.25.0 c8568f914cd2 ee46304105ca 2023-10-01T20:06:10.051 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (97s) 17s ago 22m 86.9M - 9.4.7 2c41d148cca3 e676c0004015 2023-10-01T20:06:10.051 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (2m) 0s ago 22m 57.0M - 3.5 e1d6a67b021e 7ef2818f7c68 2023-10-01T20:06:10.052 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283,8765 running (2m) 17s ago 25m 449M - 18.0.0-6505-g37d71fd3 bea6d6056140 b5797dd6a1b2 2023-10-01T20:06:10.052 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (12m) 0s ago 27m 538M - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T20:06:10.052 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (27m) 0s ago 27m 223M 2048M 17.2.0 e1d6a67b021e 2f747ad42380 2023-10-01T20:06:10.052 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (43s) 17s ago 26m 35.3M 2048M 18.0.0-6505-g37d71fd3 bea6d6056140 805778222e17 2023-10-01T20:06:10.052 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (26m) 0s ago 26m 112M 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T20:06:10.052 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (2m) 0s ago 23m 8967k - 1.5.0 0da6a335fe13 175d4990d5fd 2023-10-01T20:06:10.052 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (2m) 17s ago 23m 18.2M - 1.5.0 0da6a335fe13 b2605eeb2dd4 2023-10-01T20:06:10.052 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (25m) 0s ago 25m 174M 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T20:06:10.052 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (25m) 0s ago 25m 182M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T20:06:10.053 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (25m) 0s ago 25m 131M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T20:06:10.053 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (24m) 0s ago 24m 202M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T20:06:10.053 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (24m) 17s ago 24m 193M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T20:06:10.053 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (24m) 17s ago 24m 184M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T20:06:10.053 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (24m) 17s ago 24m 147M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T20:06:10.053 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (23m) 17s ago 23m 181M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T20:06:10.053 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (2m) 17s ago 23m 53.1M - 2.43.0 a07b618ecd1d 79b0216ed806 2023-10-01T20:06:10.053 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (22m) 0s ago 22m 100M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T20:06:10.053 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (22m) 17s ago 22m 96.8M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T20:06:10.479 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:06:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2023-10-01T20:06:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2, 2023-10-01T20:06:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 1 2023-10-01T20:06:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:06:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2023-10-01T20:06:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:06:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:06:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2023-10-01T20:06:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2023-10-01T20:06:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:06:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": {}, 2023-10-01T20:06:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "rgw": { 2023-10-01T20:06:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T20:06:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:06:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2023-10-01T20:06:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 12, 2023-10-01T20:06:10.481 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 3 2023-10-01T20:06:10.481 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T20:06:10.481 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:06:10.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:10.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: Upgrade: Target is version 18.0.0-6505-g37d71fd3 (reef) 2023-10-01T20:06:10.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24, digests ['quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24'] 2023-10-01T20:06:10.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:10.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:10.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: Upgrade: Setting container_image for all mgr 2023-10-01T20:06:10.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2023-10-01T20:06:10.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2023-10-01T20:06:10.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: Upgrade: It appears safe to stop mon.a 2023-10-01T20:06:10.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: from='client.25531 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:10.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[108286]: from='client.25537 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:10.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:10.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: Upgrade: Target is version 18.0.0-6505-g37d71fd3 (reef) 2023-10-01T20:06:10.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24, digests ['quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24'] 2023-10-01T20:06:10.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:10.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:10.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: Upgrade: Setting container_image for all mgr 2023-10-01T20:06:10.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2023-10-01T20:06:10.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2023-10-01T20:06:10.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: Upgrade: It appears safe to stop mon.a 2023-10-01T20:06:10.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: from='client.25531 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:10.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:10 smithi057 ceph-mon[114331]: from='client.25537 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:10.850 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:06:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24", 2023-10-01T20:06:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2023-10-01T20:06:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mon on host(s) smithi057", 2023-10-01T20:06:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2023-10-01T20:06:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "0/2 daemons upgraded", 2023-10-01T20:06:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mon daemons", 2023-10-01T20:06:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2023-10-01T20:06:10.851 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:06:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: Upgrade: Target is version 18.0.0-6505-g37d71fd3 (reef) 2023-10-01T20:06:10.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24, digests ['quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24'] 2023-10-01T20:06:10.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:10.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:10.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all mgr 2023-10-01T20:06:10.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2023-10-01T20:06:10.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2023-10-01T20:06:10.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: Upgrade: It appears safe to stop mon.a 2023-10-01T20:06:10.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: from='client.25531 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:10.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:10.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:10 smithi087 ceph-mon[178735]: from='client.25537 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:11.568 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/4093629984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:11.568 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[114331]: Upgrade: Updating mon.a 2023-10-01T20:06:11.568 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:11.568 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:11.568 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:11.569 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:11.569 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[114331]: Deploying daemon mon.a on smithi057 2023-10-01T20:06:11.569 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[114331]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:11.569 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[114331]: from='client.15663 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:11.569 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[108286]: from='client.? 172.21.15.57:0/4093629984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:11.569 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[108286]: Upgrade: Updating mon.a 2023-10-01T20:06:11.570 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:11.570 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:11.570 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:11.570 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[108286]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:11.570 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[108286]: Deploying daemon mon.a on smithi057 2023-10-01T20:06:11.570 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[108286]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:11.570 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-mon[108286]: from='client.15663 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:11 smithi087 ceph-mon[178735]: from='client.? 172.21.15.57:0/4093629984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:11 smithi087 ceph-mon[178735]: Upgrade: Updating mon.a 2023-10-01T20:06:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:11 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' 2023-10-01T20:06:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:11 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:11 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:11 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:11 smithi087 ceph-mon[178735]: Deploying daemon mon.a on smithi057 2023-10-01T20:06:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:11 smithi087 ceph-mon[178735]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:11.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:11 smithi087 ceph-mon[178735]: from='client.15663 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:11.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:11 smithi057 systemd[1]: Stopping Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:06:12.222 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-a[108263]: 2023-10-01T20:06:11.936+0000 7fe0b878f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:06:12.222 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-a[108263]: 2023-10-01T20:06:11.936+0000 7fe0b878f700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-01T20:06:12.223 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:11] ENGINE Bus STOPPING 2023-10-01T20:06:12.223 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-10-01T20:06:12.223 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:11] ENGINE Bus STOPPED 2023-10-01T20:06:12.223 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:11] ENGINE Bus STARTING 2023-10-01T20:06:12.223 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:12] ENGINE Serving on http://:::9283 2023-10-01T20:06:12.223 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:12 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:12] ENGINE Bus STARTED 2023-10-01T20:06:12.559 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:12 smithi057 bash[201745]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-a 2023-10-01T20:06:12.893 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:12 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-10-01T20:06:12.893 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:12 smithi057 systemd[1]: Stopped Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:06:12.894 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:12 smithi057 systemd[1]: Starting Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:06:13.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: set uid:gid to 167:167 (ceph:ceph) 2023-10-01T20:06:13.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev), process ceph-mon, pid 7 2023-10-01T20:06:13.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: pidfile_write: ignore empty --pid-file 2023-10-01T20:06:13.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: load: jerasure load: lrc 2023-10-01T20:06:13.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: RocksDB version: 7.9.2 2023-10-01T20:06:13.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Git sha 0 2023-10-01T20:06:13.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Compile date 2023-10-01 17:30:24 2023-10-01T20:06:13.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: DB SUMMARY 2023-10-01T20:06:13.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: DB Session ID: ACUHJTWKA48YK7TR94B2 2023-10-01T20:06:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: CURRENT file: CURRENT 2023-10-01T20:06:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: IDENTITY file: IDENTITY 2023-10-01T20:06:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: MANIFEST file: MANIFEST-000015 size: 3877 Bytes 2023-10-01T20:06:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000075.sst 2023-10-01T20:06:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000073.log size: 216559 ; 2023-10-01T20:06:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.error_if_exists: 0 2023-10-01T20:06:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.create_if_missing: 0 2023-10-01T20:06:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.paranoid_checks: 1 2023-10-01T20:06:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-01T20:06:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-01T20:06:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-01T20:06:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.env: 0x55682a2e9da0 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.fs: PosixFileSystem 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.info_log: 0x55682b8075a0 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_file_opening_threads: 16 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.statistics: (nil) 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.use_fsync: 0 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_log_file_size: 0 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.keep_log_file_num: 1000 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.recycle_log_file_num: 0 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.allow_fallocate: 1 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.allow_mmap_reads: 0 2023-10-01T20:06:13.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.allow_mmap_writes: 0 2023-10-01T20:06:13.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.use_direct_reads: 0 2023-10-01T20:06:13.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-01T20:06:13.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.create_missing_column_families: 0 2023-10-01T20:06:13.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.db_log_dir: 2023-10-01T20:06:13.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.wal_dir: 2023-10-01T20:06:13.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-01T20:06:13.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-01T20:06:13.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-01T20:06:13.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-01T20:06:13.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-01T20:06:13.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-01T20:06:13.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.advise_random_on_open: 1 2023-10-01T20:06:13.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.db_write_buffer_size: 0 2023-10-01T20:06:13.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.write_buffer_manager: 0x55682b7bf360 2023-10-01T20:06:13.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-01T20:06:13.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-01T20:06:13.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-01T20:06:13.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.rate_limiter: (nil) 2023-10-01T20:06:13.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-01T20:06:13.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.wal_recovery_mode: 2 2023-10-01T20:06:13.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.enable_thread_tracking: 0 2023-10-01T20:06:13.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.enable_pipelined_write: 0 2023-10-01T20:06:13.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.unordered_write: 0 2023-10-01T20:06:13.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-01T20:06:13.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-01T20:06:13.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-01T20:06:13.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-01T20:06:13.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.row_cache: None 2023-10-01T20:06:13.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.wal_filter: None 2023-10-01T20:06:13.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-01T20:06:13.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.allow_ingest_behind: 0 2023-10-01T20:06:13.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.two_write_queues: 0 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.manual_wal_flush: 0 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.wal_compression: 0 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.atomic_flush: 0 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.log_readahead_size: 0 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.best_efforts_recovery: 0 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.allow_data_in_errors: 0 2023-10-01T20:06:13.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.db_host_id: __hostname__ 2023-10-01T20:06:13.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-01T20:06:13.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_background_jobs: 2 2023-10-01T20:06:13.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_background_compactions: -1 2023-10-01T20:06:13.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_subcompactions: 1 2023-10-01T20:06:13.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-01T20:06:13.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-01T20:06:13.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-01T20:06:13.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_total_wal_size: 0 2023-10-01T20:06:13.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-01T20:06:13.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-01T20:06:13.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-01T20:06:13.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-01T20:06:13.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_open_files: -1 2023-10-01T20:06:13.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bytes_per_sync: 0 2023-10-01T20:06:13.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-01T20:06:13.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-01T20:06:13.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_readahead_size: 0 2023-10-01T20:06:13.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_background_flushes: -1 2023-10-01T20:06:13.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Compression algorithms supported: 2023-10-01T20:06:13.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-01T20:06:13.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: kZSTD supported: 0 2023-10-01T20:06:13.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: kXpressCompression supported: 0 2023-10-01T20:06:13.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: kLZ4HCCompression supported: 1 2023-10-01T20:06:13.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: kZlibCompression supported: 1 2023-10-01T20:06:13.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: kSnappyCompression supported: 1 2023-10-01T20:06:13.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: kLZ4Compression supported: 1 2023-10-01T20:06:13.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: kBZip2Compression supported: 0 2023-10-01T20:06:13.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-01T20:06:13.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-01T20:06:13.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000015 2023-10-01T20:06:13.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-01T20:06:13.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-01T20:06:13.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.merge_operator: 2023-10-01T20:06:13.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_filter: None 2023-10-01T20:06:13.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_filter_factory: None 2023-10-01T20:06:13.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.sst_partitioner_factory: None 2023-10-01T20:06:13.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-01T20:06:13.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-01T20:06:13.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55682b8076a0) 2023-10-01T20:06:13.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks: 1 2023-10-01T20:06:13.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-01T20:06:13.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-01T20:06:13.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_top_level_index_and_filter: 1 2023-10-01T20:06:13.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_type: 0 2023-10-01T20:06:13.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_index_type: 0 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_shortening: 1 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: checksum: 4 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: no_block_cache: 0 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache: 0x55682b85d1f0 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_name: BinnedLRUCache 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_options: 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: capacity : 536870912 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_shard_bits : 4 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: strict_capacity_limit : 0 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: high_pri_pool_ratio: 0.000 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_compressed: (nil) 2023-10-01T20:06:13.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: persistent_cache: (nil) 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size: 4096 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size_deviation: 10 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_restart_interval: 16 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_block_restart_interval: 1 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: metadata_block_size: 4096 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: partition_filters: 0 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: use_delta_encoding: 1 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: filter_policy: bloomfilter 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: whole_key_filtering: 1 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: verify_compression: 0 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: read_amp_bytes_per_bit: 0 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: format_version: 5 2023-10-01T20:06:13.686 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_index_compression: 1 2023-10-01T20:06:13.687 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_align: 0 2023-10-01T20:06:13.687 INFO:journalctl@ceph.mon.a.smithi057.stdout: max_auto_readahead_size: 262144 2023-10-01T20:06:13.687 INFO:journalctl@ceph.mon.a.smithi057.stdout: prepopulate_block_cache: 0 2023-10-01T20:06:13.687 INFO:journalctl@ceph.mon.a.smithi057.stdout: initial_auto_readahead_size: 8192 2023-10-01T20:06:13.687 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_file_reads_for_auto_readahead: 2 2023-10-01T20:06:13.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.write_buffer_size: 33554432 2023-10-01T20:06:13.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_write_buffer_number: 2 2023-10-01T20:06:13.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compression: NoCompression 2023-10-01T20:06:13.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bottommost_compression: Disabled 2023-10-01T20:06:13.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.prefix_extractor: nullptr 2023-10-01T20:06:13.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-01T20:06:13.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.num_levels: 7 2023-10-01T20:06:13.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-01T20:06:13.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-01T20:06:13.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-01T20:06:13.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-01T20:06:13.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-01T20:06:13.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-01T20:06:13.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-01T20:06:13.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-01T20:06:13.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-01T20:06:13.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-01T20:06:13.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-01T20:06:13.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-01T20:06:13.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-01T20:06:13.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compression_opts.level: 32767 2023-10-01T20:06:13.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compression_opts.strategy: 0 2023-10-01T20:06:13.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-01T20:06:13.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-01T20:06:13.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-01T20:06:13.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compression_opts.enabled: false 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.target_file_size_base: 67108864 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-01T20:06:13.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-01T20:06:13.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-01T20:06:13.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-01T20:06:13.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-01T20:06:13.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-01T20:06:13.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-01T20:06:13.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-01T20:06:13.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-01T20:06:13.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.arena_block_size: 1048576 2023-10-01T20:06:13.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-01T20:06:13.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-01T20:06:13.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.disable_auto_compactions: 0 2023-10-01T20:06:13.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-01T20:06:13.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-01T20:06:13.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-01T20:06:13.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-01T20:06:13.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-01T20:06:13.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-01T20:06:13.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-01T20:06:13.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-01T20:06:13.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-01T20:06:13.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-01T20:06:13.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.table_properties_collectors: 2023-10-01T20:06:13.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.inplace_update_support: 0 2023-10-01T20:06:13.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-01T20:06:13.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-01T20:06:13.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-01T20:06:13.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-01T20:06:13.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.bloom_locality: 0 2023-10-01T20:06:13.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.max_successive_merges: 0 2023-10-01T20:06:13.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-01T20:06:13.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.paranoid_file_checks: 0 2023-10-01T20:06:13.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.force_consistency_checks: 1 2023-10-01T20:06:13.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.report_bg_io_stats: 0 2023-10-01T20:06:13.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.ttl: 2592000 2023-10-01T20:06:13.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-01T20:06:13.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-01T20:06:13.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-01T20:06:13.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.enable_blob_files: false 2023-10-01T20:06:13.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.min_blob_size: 0 2023-10-01T20:06:13.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.blob_file_size: 268435456 2023-10-01T20:06:13.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-01T20:06:13.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-01T20:06:13.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-01T20:06:13.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-01T20:06:13.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-01T20:06:13.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.blob_file_starting_level: 0 2023-10-01T20:06:13.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-01T20:06:13.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 75.sst 2023-10-01T20:06:13.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2023-10-01T20:06:13.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 77, last_sequence is 35994, log_number is 73,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-01T20:06:13.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 73 2023-10-01T20:06:13.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 085fe613-ee20-4c38-a277-5a472245ad4d 2023-10-01T20:06:13.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696190773407336, "job": 1, "event": "recovery_started", "wal_files": [73]} 2023-10-01T20:06:13.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #73 mode 2 2023-10-01T20:06:13.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696190773409558, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 78, "file_size": 213984, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 35996, "largest_seqno": 36206, "table_properties": {"data_size": 212087, "index_size": 565, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 325, "raw_key_size": 2851, "raw_average_key_size": 25, "raw_value_size": 209634, "raw_average_value_size": 1871, "num_data_blocks": 25, "num_entries": 112, "num_filter_entries": 112, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696190773, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "085fe613-ee20-4c38-a277-5a472245ad4d", "db_session_id": "ACUHJTWKA48YK7TR94B2", "orig_file_number": 78, "seqno_to_time_mapping": "N/A"}} 2023-10-01T20:06:13.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696190773409663, "job": 1, "event": "recovery_finished"} 2023-10-01T20:06:13.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [db/version_set.cc:5047] Creating manifest 80 2023-10-01T20:06:13.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2023-10-01T20:06:13.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000073.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-01T20:06:13.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55682b8fa000 2023-10-01T20:06:13.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: DB pointer 0x55682b8e6000 2023-10-01T20:06:13.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-01T20:06:13.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-01T20:06:13.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** DB Stats ** 2023-10-01T20:06:13.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T20:06:13.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-01T20:06:13.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-01T20:06:13.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T20:06:13.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-01T20:06:13.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-01T20:06:13.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T20:06:13.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T20:06:13.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T20:06:13.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-01T20:06:13.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-01T20:06:13.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: L0 1/0 208.97 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-01T20:06:13.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: L6 1/0 7.95 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2023-10-01T20:06:13.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: Sum 2/0 8.15 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-01T20:06:13.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-01T20:06:13.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T20:06:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T20:06:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-01T20:06:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T20:06:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 145.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-01T20:06:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T20:06:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-01T20:06:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T20:06:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T20:06:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-01T20:06:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-01T20:06:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-01T20:06:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-01T20:06:13.699 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-01T20:06:13.699 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative compaction: 0.00 GB write, 33.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T20:06:13.699 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval compaction: 0.00 GB write, 33.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T20:06:13.699 INFO:journalctl@ceph.mon.a.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-01T20:06:13.699 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache BinnedLRUCache@0x55682b85d1f0#7 capacity: 512.00 MB usage: 1.02 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2023-10-01T20:06:13.699 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.38 KB,7.15256e-05%) IndexBlock(1,0.64 KB,0.00012219%) Misc(1,0.00 KB,0%) 2023-10-01T20:06:13.699 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T20:06:13.699 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-01T20:06:13.699 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: starting mon.a rank 0 at public addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] at bind addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T20:06:13.699 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: mon.a@-1(???) e3 preinit fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T20:06:13.699 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: mon.a@-1(???).mds e1 new map 2023-10-01T20:06:13.699 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: mon.a@-1(???).mds e1 print_map 2023-10-01T20:06:13.700 INFO:journalctl@ceph.mon.a.smithi057.stdout: e1 2023-10-01T20:06:13.700 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-01T20:06:13.700 INFO:journalctl@ceph.mon.a.smithi057.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-01T20:06:13.700 INFO:journalctl@ceph.mon.a.smithi057.stdout: legacy client fscid: -1 2023-10-01T20:06:13.700 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T20:06:13.700 INFO:journalctl@ceph.mon.a.smithi057.stdout: No filesystems configured 2023-10-01T20:06:13.700 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: mon.a@-1(???).osd e107 crush map has features 3314933000854323200, adjusting msgr requires 2023-10-01T20:06:13.700 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: mon.a@-1(???).osd e107 crush map has features 432629239337189376, adjusting msgr requires 2023-10-01T20:06:13.700 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: mon.a@-1(???).osd e107 crush map has features 432629239337189376, adjusting msgr requires 2023-10-01T20:06:13.700 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: mon.a@-1(???).osd e107 crush map has features 432629239337189376, adjusting msgr requires 2023-10-01T20:06:13.700 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 ceph-mon[202005]: mon.a@-1(???).paxosservice(auth 1..27) refresh upgraded, format 0 -> 3 2023-10-01T20:06:13.700 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 bash[201960]: 115f7a9e633089875ba84e3abba8dace96a3983b019788c71aec2a15c49de0f2 2023-10-01T20:06:13.701 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:13 smithi057 systemd[1]: Started Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:06:14.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:06:14] "GET /metrics HTTP/1.1" 200 37916 "" "Prometheus/2.43.0" 2023-10-01T20:06:16.819 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:06:16.820 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:06:16.820 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:06:16.820 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:06:16.820 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:16.820 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:16.820 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:16.820 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: mon.a calling monitor election 2023-10-01T20:06:16.820 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T20:06:16.820 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} removed_ranks: {} 2023-10-01T20:06:16.820 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: fsmap 2023-10-01T20:06:16.821 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: osdmap e107: 8 total, 8 up, 8 in 2023-10-01T20:06:16.821 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: mgrmap e43: y(active, since 2m), standbys: x 2023-10-01T20:06:16.821 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: from='mgr.25171 ' entity='' 2023-10-01T20:06:16.821 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: mgrmap e44: y(active, since 2m), standbys: x 2023-10-01T20:06:16.821 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: overall HEALTH_OK 2023-10-01T20:06:16.821 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mon[178735]: from='mgr.25171 ' entity='' 2023-10-01T20:06:16.821 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: ignoring --setuser ceph since I am not root 2023-10-01T20:06:16.821 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: ignoring --setgroup ceph since I am not root 2023-10-01T20:06:16.822 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-mgr[162810]: -- 172.21.15.87:0/1644302076 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5605b15cb1e0 con 0x5605b15a1000 2023-10-01T20:06:16.822 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:16.648+0000 7f01cb960200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T20:06:16.822 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:16 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:16.818+0000 7f01cb960200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T20:06:16.877 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ignoring --setuser ceph since I am not root 2023-10-01T20:06:16.878 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ignoring --setgroup ceph since I am not root 2023-10-01T20:06:16.878 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mgr[161953]: -- 172.21.15.57:0/2232301895 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55773461f1e0 con 0x5577345f4c00 2023-10-01T20:06:16.878 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:16.675+0000 7fa509be2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-01T20:06:16.878 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:16.874+0000 7fa509be2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-01T20:06:16.878 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:06:16.878 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:06:16.878 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:06:16.879 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:06:16.879 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:16.879 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:16.879 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:16.879 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: mon.a calling monitor election 2023-10-01T20:06:16.879 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T20:06:16.879 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} removed_ranks: {} 2023-10-01T20:06:16.879 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: fsmap 2023-10-01T20:06:16.879 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: osdmap e107: 8 total, 8 up, 8 in 2023-10-01T20:06:16.879 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: mgrmap e43: y(active, since 2m), standbys: x 2023-10-01T20:06:16.879 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: from='mgr.25171 ' entity='' 2023-10-01T20:06:16.879 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: mgrmap e44: y(active, since 2m), standbys: x 2023-10-01T20:06:16.880 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: overall HEALTH_OK 2023-10-01T20:06:16.880 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[202005]: from='mgr.25171 ' entity='' 2023-10-01T20:06:16.880 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:06:16.880 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:06:16.880 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:06:16.880 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: from='mgr.25171 172.21.15.57:0/2577580046' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:06:16.880 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:16.880 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:16.880 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:16.880 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: mon.a calling monitor election 2023-10-01T20:06:16.881 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T20:06:16.881 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} removed_ranks: {} 2023-10-01T20:06:16.881 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: fsmap 2023-10-01T20:06:16.881 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: osdmap e107: 8 total, 8 up, 8 in 2023-10-01T20:06:16.881 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: mgrmap e43: y(active, since 2m), standbys: x 2023-10-01T20:06:16.881 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: from='mgr.25171 ' entity='' 2023-10-01T20:06:16.881 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: mgrmap e44: y(active, since 2m), standbys: x 2023-10-01T20:06:16.881 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: overall HEALTH_OK 2023-10-01T20:06:16.881 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:16 smithi057 ceph-mon[114331]: from='mgr.25171 ' entity='' 2023-10-01T20:06:17.633 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:17 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:17.147+0000 7f01cb960200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T20:06:17.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:17 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:17.247+0000 7fa509be2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-01T20:06:18.081 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:17 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:17.820+0000 7fa509be2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T20:06:18.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:17 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:17.646+0000 7f01cb960200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-01T20:06:18.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:17 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:17.879+0000 7f01cb960200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-01T20:06:18.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:18 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:18.080+0000 7fa509be2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-01T20:06:18.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:18 smithi087 ceph-mon[178735]: mgrmap e45: y(active, since 2m), standbys: x 2023-10-01T20:06:18.633 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:18 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:18.302+0000 7f01cb960200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T20:06:18.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:18 smithi057 ceph-mon[202005]: mgrmap e45: y(active, since 2m), standbys: x 2023-10-01T20:06:18.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:18 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:18.564+0000 7fa509be2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-01T20:06:18.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:18 smithi057 ceph-mon[114331]: mgrmap e45: y(active, since 2m), standbys: x 2023-10-01T20:06:19.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:19.021+0000 7f01cb960200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T20:06:19.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:19.110+0000 7f01cb960200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T20:06:19.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:19 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:19.382+0000 7fa509be2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-01T20:06:19.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:19 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:19.482+0000 7fa509be2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-01T20:06:20.123 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:19.719+0000 7f01cb960200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T20:06:20.123 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:19.808+0000 7f01cb960200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T20:06:20.123 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:19 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:19.897+0000 7f01cb960200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T20:06:20.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:20.122+0000 7f01cb960200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T20:06:20.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:20.216+0000 7f01cb960200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T20:06:20.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:20.315+0000 7f01cb960200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T20:06:20.651 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:20 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:20.184+0000 7fa509be2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-01T20:06:20.651 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:20 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:20.283+0000 7fa509be2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-01T20:06:20.651 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:20 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:20.385+0000 7fa509be2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-01T20:06:20.822 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:20.535+0000 7f01cb960200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T20:06:20.822 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:20.632+0000 7f01cb960200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T20:06:20.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:20 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:20.650+0000 7fa509be2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-01T20:06:20.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:20 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:20.757+0000 7fa509be2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-01T20:06:20.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:20 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:20.867+0000 7fa509be2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-01T20:06:21.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:20.821+0000 7f01cb960200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T20:06:21.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:20 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:20.910+0000 7f01cb960200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T20:06:21.421 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:21 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:21.125+0000 7fa509be2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-01T20:06:21.422 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:21 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:21.238+0000 7fa509be2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-01T20:06:21.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:21 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:21.456+0000 7fa509be2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-01T20:06:21.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:21 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:21.558+0000 7fa509be2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-01T20:06:22.060 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:21 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:21.670+0000 7f01cb960200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T20:06:22.060 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:21 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:21.872+0000 7f01cb960200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T20:06:22.383 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:22 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:22.059+0000 7f01cb960200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T20:06:22.882 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:22.439+0000 7fa509be2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-01T20:06:22.882 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:22.668+0000 7fa509be2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-01T20:06:23.110 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:22 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:22.727+0000 7f01cb960200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T20:06:23.110 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:22 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:22.920+0000 7f01cb960200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T20:06:23.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:22 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:22.880+0000 7fa509be2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-01T20:06:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-mon[178735]: Standby manager daemon x restarted 2023-10-01T20:06:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-mon[178735]: Standby manager daemon x started 2023-10-01T20:06:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-mon[178735]: from='mgr.? 172.21.15.87:0/1248557907' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T20:06:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-mon[178735]: from='mgr.? 172.21.15.87:0/1248557907' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T20:06:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-mon[178735]: from='mgr.? 172.21.15.87:0/1248557907' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T20:06:23.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-mon[178735]: from='mgr.? 172.21.15.87:0/1248557907' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T20:06:23.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:23.109+0000 7f01cb960200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T20:06:23.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:23.232+0000 7f01cb960200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T20:06:23.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: [01/Oct/2023:20:06:23] ENGINE Bus STARTING 2023-10-01T20:06:23.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: CherryPy Checker: 2023-10-01T20:06:23.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: The Application mounted at '' has an empty config. 2023-10-01T20:06:23.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: 2023-10-01T20:06:23.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: [01/Oct/2023:20:06:23] ENGINE Serving on http://:::9283 2023-10-01T20:06:23.384 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:06:23 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x[162786]: [01/Oct/2023:20:06:23] ENGINE Bus STARTED 2023-10-01T20:06:23.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-mon[202005]: Standby manager daemon x restarted 2023-10-01T20:06:23.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-mon[202005]: Standby manager daemon x started 2023-10-01T20:06:23.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-mon[202005]: from='mgr.? 172.21.15.87:0/1248557907' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T20:06:23.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-mon[202005]: from='mgr.? 172.21.15.87:0/1248557907' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T20:06:23.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-mon[202005]: from='mgr.? 172.21.15.87:0/1248557907' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T20:06:23.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-mon[202005]: from='mgr.? 172.21.15.87:0/1248557907' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T20:06:23.657 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-mon[114331]: Standby manager daemon x restarted 2023-10-01T20:06:23.657 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-mon[114331]: Standby manager daemon x started 2023-10-01T20:06:23.657 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/1248557907' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-10-01T20:06:23.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/1248557907' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-01T20:06:23.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/1248557907' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-10-01T20:06:23.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-mon[114331]: from='mgr.? 172.21.15.87:0/1248557907' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-01T20:06:23.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:23.655+0000 7fa509be2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-01T20:06:23.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:23 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:23.874+0000 7fa509be2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-01T20:06:24.355 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:24.091+0000 7fa509be2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-01T20:06:24.355 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:24.229+0000 7fa509be2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-01T20:06:24.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: mgrmap e46: y(active, since 2m), standbys: x 2023-10-01T20:06:24.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: Active manager daemon y restarted 2023-10-01T20:06:24.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: Activating manager daemon y 2023-10-01T20:06:24.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: osdmap e108: 8 total, 8 up, 8 in 2023-10-01T20:06:24.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: mgrmap e47: y(active, starting, since 0.00857595s), standbys: x 2023-10-01T20:06:24.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:06:24.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:06:24.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:06:24.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T20:06:24.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T20:06:24.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:06:24.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T20:06:24.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T20:06:24.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T20:06:24.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T20:06:24.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T20:06:24.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T20:06:24.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T20:06:24.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T20:06:24.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T20:06:24.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T20:06:24.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[202005]: Manager daemon y is now available 2023-10-01T20:06:24.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-c[114308]: 2023-10-01T20:06:24.251+0000 7fdce1b15700 -1 mon.c@1(peon).osd e108 update_from_paxos full map CRC mismatch, resetting to canonical 2023-10-01T20:06:24.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: mgrmap e46: y(active, since 2m), standbys: x 2023-10-01T20:06:24.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: Active manager daemon y restarted 2023-10-01T20:06:24.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: Activating manager daemon y 2023-10-01T20:06:24.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: osdmap e108: 8 total, 8 up, 8 in 2023-10-01T20:06:24.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: mgrmap e47: y(active, starting, since 0.00857595s), standbys: x 2023-10-01T20:06:24.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:06:24.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:06:24.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:06:24.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T20:06:24.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T20:06:24.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:06:24.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T20:06:24.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T20:06:24.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T20:06:24.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T20:06:24.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T20:06:24.359 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T20:06:24.359 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T20:06:24.359 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T20:06:24.359 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T20:06:24.359 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T20:06:24.359 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-mon[114331]: Manager daemon y is now available 2023-10-01T20:06:24.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: mgrmap e46: y(active, since 2m), standbys: x 2023-10-01T20:06:24.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: Active manager daemon y restarted 2023-10-01T20:06:24.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: Activating manager daemon y 2023-10-01T20:06:24.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: osdmap e108: 8 total, 8 up, 8 in 2023-10-01T20:06:24.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: mgrmap e47: y(active, starting, since 0.00857595s), standbys: x 2023-10-01T20:06:24.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:06:24.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:06:24.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:06:24.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-01T20:06:24.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-01T20:06:24.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:06:24.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-01T20:06:24.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-01T20:06:24.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-01T20:06:24.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-01T20:06:24.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-01T20:06:24.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-01T20:06:24.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-01T20:06:24.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-01T20:06:24.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-01T20:06:24.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-01T20:06:24.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:24 smithi087 ceph-mon[178735]: Manager daemon y is now available 2023-10-01T20:06:24.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:24] ENGINE Bus STARTING 2023-10-01T20:06:24.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: CherryPy Checker: 2023-10-01T20:06:24.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: The Application mounted at '' has an empty config. 2023-10-01T20:06:24.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:24.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:24] ENGINE Serving on http://:::9283 2023-10-01T20:06:24.672 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:24] ENGINE Bus STARTED 2023-10-01T20:06:25.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:06:24] "GET /metrics HTTP/1.1" 200 35115 "" "Prometheus/2.43.0" 2023-10-01T20:06:25.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: failed to encode map e108 with expected crc 2023-10-01T20:06:25.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: failed to encode map e108 with expected crc 2023-10-01T20:06:25.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: failed to encode map e108 with expected crc 2023-10-01T20:06:25.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:25.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:25.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:25.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:06:25.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:06:25.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:25.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:25.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T20:06:25.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:06:25.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T20:06:25.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T20:06:25.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: failed to encode map e108 with expected crc 2023-10-01T20:06:25.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T20:06:25.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:25 smithi087 ceph-mon[178735]: mgrmap e48: y(active, since 1.01478s), standbys: x 2023-10-01T20:06:25.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:25.266+0000 7fa4cc645700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-10-01T20:06:25.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: failed to encode map e108 with expected crc 2023-10-01T20:06:25.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: failed to encode map e108 with expected crc 2023-10-01T20:06:25.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: failed to encode map e108 with expected crc 2023-10-01T20:06:25.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:25.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:25.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:25.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:06:25.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:06:25.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T20:06:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:06:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T20:06:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T20:06:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: failed to encode map e108 with expected crc 2023-10-01T20:06:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T20:06:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[202005]: mgrmap e48: y(active, since 1.01478s), standbys: x 2023-10-01T20:06:25.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: failed to encode map e108 with expected crc 2023-10-01T20:06:25.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: failed to encode map e108 with expected crc 2023-10-01T20:06:25.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: failed to encode map e108 with expected crc 2023-10-01T20:06:25.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:25.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:25.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:25.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:06:25.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-10-01T20:06:25.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:25.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:25.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-01T20:06:25.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:06:25.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.conf 2023-10-01T20:06:25.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-10-01T20:06:25.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: failed to encode map e108 with expected crc 2023-10-01T20:06:25.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-10-01T20:06:25.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:25 smithi057 ceph-mon[114331]: mgrmap e48: y(active, since 1.01478s), standbys: x 2023-10-01T20:06:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: [01/Oct/2023:20:06:24] ENGINE Bus STARTING 2023-10-01T20:06:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:06:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:06:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: [01/Oct/2023:20:06:25] ENGINE Serving on http://172.21.15.57:8765 2023-10-01T20:06:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: [01/Oct/2023:20:06:25] ENGINE Serving on https://172.21.15.57:7151 2023-10-01T20:06:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: [01/Oct/2023:20:06:25] ENGINE Bus STARTED 2023-10-01T20:06:26.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:06:26.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:26.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:06:26.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:06:26.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:06:26.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:26 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T20:06:26.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: [01/Oct/2023:20:06:24] ENGINE Bus STARTING 2023-10-01T20:06:26.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:06:26.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:06:26.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: [01/Oct/2023:20:06:25] ENGINE Serving on http://172.21.15.57:8765 2023-10-01T20:06:26.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: [01/Oct/2023:20:06:25] ENGINE Serving on https://172.21.15.57:7151 2023-10-01T20:06:26.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: [01/Oct/2023:20:06:25] ENGINE Bus STARTED 2023-10-01T20:06:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:06:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:06:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:06:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:06:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T20:06:26.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: [01/Oct/2023:20:06:24] ENGINE Bus STARTING 2023-10-01T20:06:26.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:06:26.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-01T20:06:26.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: [01/Oct/2023:20:06:25] ENGINE Serving on http://172.21.15.57:8765 2023-10-01T20:06:26.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: [01/Oct/2023:20:06:25] ENGINE Serving on https://172.21.15.57:7151 2023-10-01T20:06:26.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: [01/Oct/2023:20:06:25] ENGINE Bus STARTED 2023-10-01T20:06:26.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:06:26.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:26.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:06:26.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-01T20:06:26.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: Updating smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:06:26.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:26.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:26 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi057", "caps": []}]: dispatch 2023-10-01T20:06:27.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:27 smithi057 ceph-mon[202005]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:06:27.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:27 smithi057 ceph-mon[202005]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:06:27.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:27 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T20:06:27.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:27 smithi057 ceph-mon[202005]: mgrmap e49: y(active, since 2s), standbys: x 2023-10-01T20:06:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:27 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:27.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:27 smithi057 ceph-mon[114331]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:06:27.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:27 smithi057 ceph-mon[114331]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:06:27.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:27 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T20:06:27.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:27 smithi057 ceph-mon[114331]: mgrmap e49: y(active, since 2s), standbys: x 2023-10-01T20:06:27.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:27 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:27.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:27 smithi087 ceph-mon[178735]: Updating smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-01T20:06:27.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:27 smithi087 ceph-mon[178735]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:06:27.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:27 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T20:06:27.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:27 smithi087 ceph-mon[178735]: mgrmap e49: y(active, since 2s), standbys: x 2023-10-01T20:06:27.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:27 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:28.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:28.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:28.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T20:06:28.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:28 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:28 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:28.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:28 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:28.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:28 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:28.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:28 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T20:06:28.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:28 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:28 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:28 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:28 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:28 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi087", "caps": []}]: dispatch 2023-10-01T20:06:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:28 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.618 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:06:29.618 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.618 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.618 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.618 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:29.618 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.619 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.619 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.619 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.619 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.619 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.619 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.619 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.619 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.619 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.619 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.619 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.619 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: mgrmap e50: y(active, since 4s), standbys: x 2023-10-01T20:06:29.621 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.621 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.621 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.621 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.621 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.621 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.621 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.621 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.621 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.621 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.621 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.621 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.622 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.622 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:29 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.843 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:06:29.843 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.844 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.844 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.844 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:29.844 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.844 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.845 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.845 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.845 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.845 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.845 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.845 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.846 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.846 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.846 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.846 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.846 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.846 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.847 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.847 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.847 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.847 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.847 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.847 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.847 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.848 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.848 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2023-10-01T20:06:29.849 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.849 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.849 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.850 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:29.850 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.850 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.850 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.850 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.850 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.852 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.852 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.852 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.852 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.852 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.852 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.853 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.853 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.853 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.853 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.854 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.854 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.854 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: mgrmap e50: y(active, since 4s), standbys: x 2023-10-01T20:06:29.854 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.854 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.855 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.855 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.855 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.855 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.855 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.855 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.855 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.856 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.856 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.856 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.856 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.856 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.856 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: mgrmap e50: y(active, since 4s), standbys: x 2023-10-01T20:06:29.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.858 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.858 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:29.858 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.858 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:29.858 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:29.858 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:29.858 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:29.859 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:29 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2023-10-01T20:06:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2023-10-01T20:06:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:30.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2023-10-01T20:06:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2023-10-01T20:06:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:30 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:30.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:30.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:30.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:30.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:30.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:30.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:30.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:30 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:31 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:31.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:31 smithi087 ceph-mon[178735]: Upgrade: It appears safe to stop mon.a 2023-10-01T20:06:31.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:31 smithi087 ceph-mon[178735]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2023-10-01T20:06:31.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:31 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:31 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:31 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:31 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:31 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:31 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[202005]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[202005]: Upgrade: It appears safe to stop mon.a 2023-10-01T20:06:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[202005]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2023-10-01T20:06:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:31.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[114331]: Upgrade: It appears safe to stop mon.a 2023-10-01T20:06:31.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[114331]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2023-10-01T20:06:31.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:31.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:31 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:32.812 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:32 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:32.812 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:32 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:32.812 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:32 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:32.813 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:32 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:32.813 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:32 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:32.813 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:32 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:32.813 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:32 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:32.813 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:32 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:32.813 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:32 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:32.813 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:32 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:32.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:32 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:32.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:32 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:32.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:32 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:32.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:32 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:32.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:32 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:33.604 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:33 smithi087 ceph-mon[178735]: Upgrade: Updating mon.a 2023-10-01T20:06:33.604 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:33 smithi087 ceph-mon[178735]: Deploying daemon mon.a on smithi057 2023-10-01T20:06:33.604 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:33 smithi087 ceph-mon[178735]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T20:06:33.604 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:33 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:33.604 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:33 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:33.628 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:33 smithi057 systemd[1]: Stopping Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:06:33.628 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-mon[202005]: Upgrade: Updating mon.a 2023-10-01T20:06:33.628 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-mon[202005]: Deploying daemon mon.a on smithi057 2023-10-01T20:06:33.628 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-mon[202005]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T20:06:33.628 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:33.628 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-mon[202005]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:33.628 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-a[201982]: 2023-10-01T20:06:33.623+0000 7ff221591700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:06:33.628 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-a[201982]: 2023-10-01T20:06:33.623+0000 7ff221591700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-01T20:06:33.629 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-mon[114331]: Upgrade: Updating mon.a 2023-10-01T20:06:33.629 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-mon[114331]: Deploying daemon mon.a on smithi057 2023-10-01T20:06:33.629 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-mon[114331]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-10-01T20:06:33.629 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:33.629 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' 2023-10-01T20:06:33.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:33 smithi057 bash[205123]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-a 2023-10-01T20:06:33.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:33] ENGINE Bus STOPPING 2023-10-01T20:06:33.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:33] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-10-01T20:06:33.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:33] ENGINE Bus STOPPED 2023-10-01T20:06:33.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:33] ENGINE Bus STARTING 2023-10-01T20:06:33.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:33] ENGINE Serving on http://:::9283 2023-10-01T20:06:33.923 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:33 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: [01/Oct/2023:20:06:33] ENGINE Bus STARTED 2023-10-01T20:06:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-10-01T20:06:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 systemd[1]: Stopped Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:06:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 systemd[1]: Starting Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:06:34.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:06:34] "GET /metrics HTTP/1.1" 200 35115 "" "Prometheus/2.43.0" 2023-10-01T20:06:34.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: set uid:gid to 167:167 (ceph:ceph) 2023-10-01T20:06:34.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev), process ceph-mon, pid 7 2023-10-01T20:06:34.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: pidfile_write: ignore empty --pid-file 2023-10-01T20:06:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 bash[205351]: 99dcb8cd4b2ab0640b73422ae34fc2e7eab4da4c20bba289ed21ac6ec504d08c 2023-10-01T20:06:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: load: jerasure load: lrc 2023-10-01T20:06:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: RocksDB version: 7.9.2 2023-10-01T20:06:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Git sha 0 2023-10-01T20:06:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Compile date 2023-10-01 17:30:24 2023-10-01T20:06:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: DB SUMMARY 2023-10-01T20:06:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: DB Session ID: GFR8ONF4UCNWNQH2TJYR 2023-10-01T20:06:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: CURRENT file: CURRENT 2023-10-01T20:06:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: IDENTITY file: IDENTITY 2023-10-01T20:06:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: MANIFEST file: MANIFEST-000080 size: 277 Bytes 2023-10-01T20:06:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 2, files: 000075.sst 000078.sst 2023-10-01T20:06:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000079.log size: 2568070 ; 2023-10-01T20:06:34.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.error_if_exists: 0 2023-10-01T20:06:34.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.create_if_missing: 0 2023-10-01T20:06:34.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.paranoid_checks: 1 2023-10-01T20:06:34.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-01T20:06:34.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-01T20:06:34.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-01T20:06:34.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.env: 0x56401c0a5da0 2023-10-01T20:06:34.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.fs: PosixFileSystem 2023-10-01T20:06:34.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.info_log: 0x56401d1d75a0 2023-10-01T20:06:34.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_file_opening_threads: 16 2023-10-01T20:06:34.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.statistics: (nil) 2023-10-01T20:06:34.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.use_fsync: 0 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_log_file_size: 0 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.keep_log_file_num: 1000 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.recycle_log_file_num: 0 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.allow_fallocate: 1 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.allow_mmap_reads: 0 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.allow_mmap_writes: 0 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.use_direct_reads: 0 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.create_missing_column_families: 0 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.db_log_dir: 2023-10-01T20:06:34.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.wal_dir: 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.advise_random_on_open: 1 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.db_write_buffer_size: 0 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.write_buffer_manager: 0x56401d18f360 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-01T20:06:34.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.rate_limiter: (nil) 2023-10-01T20:06:34.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-01T20:06:34.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.wal_recovery_mode: 2 2023-10-01T20:06:34.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.enable_thread_tracking: 0 2023-10-01T20:06:34.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.enable_pipelined_write: 0 2023-10-01T20:06:34.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.unordered_write: 0 2023-10-01T20:06:34.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-01T20:06:34.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-01T20:06:34.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-01T20:06:34.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-01T20:06:34.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.row_cache: None 2023-10-01T20:06:34.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.wal_filter: None 2023-10-01T20:06:34.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-01T20:06:34.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.allow_ingest_behind: 0 2023-10-01T20:06:34.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.two_write_queues: 0 2023-10-01T20:06:34.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.manual_wal_flush: 0 2023-10-01T20:06:34.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.wal_compression: 0 2023-10-01T20:06:34.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.atomic_flush: 0 2023-10-01T20:06:34.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-01T20:06:34.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-01T20:06:34.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-01T20:06:34.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.log_readahead_size: 0 2023-10-01T20:06:34.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-01T20:06:34.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.best_efforts_recovery: 0 2023-10-01T20:06:34.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-01T20:06:34.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-01T20:06:34.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.allow_data_in_errors: 0 2023-10-01T20:06:34.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.db_host_id: __hostname__ 2023-10-01T20:06:34.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-01T20:06:34.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_background_jobs: 2 2023-10-01T20:06:34.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_background_compactions: -1 2023-10-01T20:06:34.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_subcompactions: 1 2023-10-01T20:06:34.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_total_wal_size: 0 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_open_files: -1 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bytes_per_sync: 0 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_readahead_size: 0 2023-10-01T20:06:34.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_background_flushes: -1 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Compression algorithms supported: 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: kZSTD supported: 0 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: kXpressCompression supported: 0 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: kLZ4HCCompression supported: 1 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: kZlibCompression supported: 1 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: kSnappyCompression supported: 1 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: kLZ4Compression supported: 1 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: kBZip2Compression supported: 0 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000080 2023-10-01T20:06:34.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-01T20:06:34.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-01T20:06:34.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.merge_operator: 2023-10-01T20:06:34.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_filter: None 2023-10-01T20:06:34.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_filter_factory: None 2023-10-01T20:06:34.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.sst_partitioner_factory: None 2023-10-01T20:06:34.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-01T20:06:34.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-01T20:06:34.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56401d1d76a0) 2023-10-01T20:06:34.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks: 1 2023-10-01T20:06:34.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-01T20:06:34.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-01T20:06:34.934 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_top_level_index_and_filter: 1 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_type: 0 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_index_type: 0 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_shortening: 1 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: checksum: 4 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: no_block_cache: 0 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache: 0x56401d22d1f0 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_name: BinnedLRUCache 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_options: 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: capacity : 536870912 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_shard_bits : 4 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: strict_capacity_limit : 0 2023-10-01T20:06:34.935 INFO:journalctl@ceph.mon.a.smithi057.stdout: high_pri_pool_ratio: 0.000 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_compressed: (nil) 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: persistent_cache: (nil) 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size: 4096 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size_deviation: 10 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_restart_interval: 16 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_block_restart_interval: 1 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: metadata_block_size: 4096 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: partition_filters: 0 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: use_delta_encoding: 1 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: filter_policy: bloomfilter 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: whole_key_filtering: 1 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: verify_compression: 0 2023-10-01T20:06:34.936 INFO:journalctl@ceph.mon.a.smithi057.stdout: read_amp_bytes_per_bit: 0 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout: format_version: 5 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_index_compression: 1 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_align: 0 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout: max_auto_readahead_size: 262144 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout: prepopulate_block_cache: 0 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout: initial_auto_readahead_size: 8192 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_file_reads_for_auto_readahead: 2 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.write_buffer_size: 33554432 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_write_buffer_number: 2 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compression: NoCompression 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bottommost_compression: Disabled 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.prefix_extractor: nullptr 2023-10-01T20:06:34.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-01T20:06:34.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.num_levels: 7 2023-10-01T20:06:34.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-01T20:06:34.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-01T20:06:34.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-01T20:06:34.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-01T20:06:34.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-01T20:06:34.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-01T20:06:34.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-01T20:06:34.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-01T20:06:34.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-01T20:06:34.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-01T20:06:34.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compression_opts.level: 32767 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compression_opts.strategy: 0 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compression_opts.enabled: false 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-01T20:06:34.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.target_file_size_base: 67108864 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-01T20:06:34.940 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-01T20:06:34.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-01T20:06:34.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-01T20:06:34.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.arena_block_size: 1048576 2023-10-01T20:06:34.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-01T20:06:34.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-01T20:06:34.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.disable_auto_compactions: 0 2023-10-01T20:06:34.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-01T20:06:34.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-01T20:06:34.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-01T20:06:34.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-01T20:06:34.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-01T20:06:34.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.table_properties_collectors: 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.inplace_update_support: 0 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.bloom_locality: 0 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.max_successive_merges: 0 2023-10-01T20:06:34.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-01T20:06:34.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.paranoid_file_checks: 0 2023-10-01T20:06:34.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.force_consistency_checks: 1 2023-10-01T20:06:34.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.report_bg_io_stats: 0 2023-10-01T20:06:34.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.ttl: 2592000 2023-10-01T20:06:34.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-01T20:06:34.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-01T20:06:34.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-01T20:06:34.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.enable_blob_files: false 2023-10-01T20:06:34.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.min_blob_size: 0 2023-10-01T20:06:34.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.blob_file_size: 268435456 2023-10-01T20:06:34.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-01T20:06:34.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-01T20:06:34.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-01T20:06:34.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-01T20:06:34.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-01T20:06:34.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.blob_file_starting_level: 0 2023-10-01T20:06:34.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-01T20:06:34.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 75.sst 2023-10-01T20:06:34.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 systemd[1]: Started Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:06:34.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2023-10-01T20:06:34.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000080 succeeded,manifest_file_number is 80, next_file_number is 82, last_sequence is 36206, log_number is 74,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 74 2023-10-01T20:06:34.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 74 2023-10-01T20:06:34.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 085fe613-ee20-4c38-a277-5a472245ad4d 2023-10-01T20:06:34.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696190794671908, "job": 1, "event": "recovery_started", "wal_files": [79]} 2023-10-01T20:06:34.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #79 mode 2 2023-10-01T20:06:34.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696190794681069, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 83, "file_size": 2398910, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 36209, "largest_seqno": 36989, "table_properties": {"data_size": 2394240, "index_size": 2628, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1029, "raw_key_size": 9019, "raw_average_key_size": 23, "raw_value_size": 2385968, "raw_average_value_size": 6197, "num_data_blocks": 119, "num_entries": 385, "num_filter_entries": 385, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696190794, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "085fe613-ee20-4c38-a277-5a472245ad4d", "db_session_id": "GFR8ONF4UCNWNQH2TJYR", "orig_file_number": 83, "seqno_to_time_mapping": "N/A"}} 2023-10-01T20:06:34.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696190794681178, "job": 1, "event": "recovery_finished"} 2023-10-01T20:06:34.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [db/version_set.cc:5047] Creating manifest 85 2023-10-01T20:06:34.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2023-10-01T20:06:34.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000079.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-01T20:06:34.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56401d2ca000 2023-10-01T20:06:34.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: DB pointer 0x56401d2b6000 2023-10-01T20:06:34.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-01T20:06:34.946 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-01T20:06:34.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** DB Stats ** 2023-10-01T20:06:34.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T20:06:34.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-01T20:06:34.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-01T20:06:34.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T20:06:34.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-01T20:06:34.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-01T20:06:34.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T20:06:34.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T20:06:34.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T20:06:34.946 INFO:journalctl@ceph.mon.a.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-01T20:06:34.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-01T20:06:34.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: L0 2/0 2.49 MB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 468.9 0.00 0.00 1 0.005 0 0 0.0 0.0 2023-10-01T20:06:34.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: L6 1/0 7.95 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2023-10-01T20:06:34.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: Sum 3/0 10.44 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 468.9 0.00 0.00 1 0.005 0 0 0.0 0.0 2023-10-01T20:06:34.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 468.9 0.00 0.00 1 0.005 0 0 0.0 0.0 2023-10-01T20:06:34.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T20:06:34.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T20:06:34.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-01T20:06:34.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T20:06:34.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 468.9 0.00 0.00 1 0.005 0 0 0.0 0.0 2023-10-01T20:06:34.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T20:06:34.947 INFO:journalctl@ceph.mon.a.smithi057.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: Flush(GB): cumulative 0.002, interval 0.002 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative compaction: 0.00 GB write, 173.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval compaction: 0.00 GB write, 173.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache BinnedLRUCache@0x56401d22d1f0#7 capacity: 512.00 MB usage: 5.00 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.2e-05 secs_since: 0 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.53 KB,0.000292063%) IndexBlock(2,3.47 KB,0.000661612%) Misc(1,0.00 KB,0%) 2023-10-01T20:06:34.948 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T20:06:34.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-01T20:06:34.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: starting mon.a rank 0 at public addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] at bind addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T20:06:34.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: mon.a@-1(???) e3 preinit fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T20:06:34.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: mon.a@-1(???).mds e1 new map 2023-10-01T20:06:34.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: mon.a@-1(???).mds e1 print_map 2023-10-01T20:06:34.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: e1 2023-10-01T20:06:34.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-01T20:06:34.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-01T20:06:34.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: legacy client fscid: -1 2023-10-01T20:06:34.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-01T20:06:34.949 INFO:journalctl@ceph.mon.a.smithi057.stdout: No filesystems configured 2023-10-01T20:06:34.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: mon.a@-1(???).osd e108 crush map has features 3314933000854323200, adjusting msgr requires 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: mon.a@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: mon.a@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: mon.a@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: mon.a@-1(???).paxosservice(auth 1..28) refresh upgraded, format 0 -> 3 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: expand_channel_meta expand map: {default=false} 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: expand_channel_meta from 'false' to 'false' 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: expand_channel_meta expanded map: {default=false} 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: expand_channel_meta expand map: {default=info} 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: expand_channel_meta from 'info' to 'info' 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: expand_channel_meta expanded map: {default=info} 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: expand_channel_meta expand map: {default=daemon} 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: expand_channel_meta from 'daemon' to 'daemon' 2023-10-01T20:06:34.950 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: expand_channel_meta expanded map: {default=daemon} 2023-10-01T20:06:34.951 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: expand_channel_meta expand map: {default=debug} 2023-10-01T20:06:34.951 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: expand_channel_meta from 'debug' to 'debug' 2023-10-01T20:06:34.951 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:34 smithi057 ceph-mon[205388]: expand_channel_meta expanded map: {default=debug} 2023-10-01T20:06:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:06:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:06:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:06:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2023-10-01T20:06:38.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T20:06:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: mon.a calling monitor election 2023-10-01T20:06:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T20:06:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} removed_ranks: {} 2023-10-01T20:06:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: fsmap 2023-10-01T20:06:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: osdmap e108: 8 total, 8 up, 8 in 2023-10-01T20:06:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: mgrmap e50: y(active, since 13s), standbys: x 2023-10-01T20:06:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='' 2023-10-01T20:06:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: overall HEALTH_OK 2023-10-01T20:06:38.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='' 2023-10-01T20:06:38.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='' 2023-10-01T20:06:38.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:37 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:06:38.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:06:38.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:06:38.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2023-10-01T20:06:38.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:38.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T20:06:38.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: mon.a calling monitor election 2023-10-01T20:06:38.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T20:06:38.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} removed_ranks: {} 2023-10-01T20:06:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: fsmap 2023-10-01T20:06:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: osdmap e108: 8 total, 8 up, 8 in 2023-10-01T20:06:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: mgrmap e50: y(active, since 13s), standbys: x 2023-10-01T20:06:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='' 2023-10-01T20:06:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: overall HEALTH_OK 2023-10-01T20:06:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='' 2023-10-01T20:06:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='' 2023-10-01T20:06:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:06:38.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:06:38.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:06:38.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2023-10-01T20:06:38.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:38.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-01T20:06:38.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: mon.a calling monitor election 2023-10-01T20:06:38.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T20:06:38.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} removed_ranks: {} 2023-10-01T20:06:38.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: fsmap 2023-10-01T20:06:38.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: osdmap e108: 8 total, 8 up, 8 in 2023-10-01T20:06:38.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: mgrmap e50: y(active, since 13s), standbys: x 2023-10-01T20:06:38.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='' 2023-10-01T20:06:38.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: overall HEALTH_OK 2023-10-01T20:06:38.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='' 2023-10-01T20:06:38.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='' 2023-10-01T20:06:38.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:38.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:37 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.029 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.029 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.029 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T20:06:39.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T20:06:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:38 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T20:06:39.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:39.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:39.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:39.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:39.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:39.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:39.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:38 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:06:40.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:39 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:06:40.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:06:40.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:39 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T20:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:40.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:40.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:40.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:40.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:40.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:40.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:40 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:41.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:41.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:41.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:41.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:41.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:41.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:41.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:41.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:41.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:41.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:41.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:41.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:41.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:41.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:41.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:41.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:41.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:41.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2023-10-01T20:06:41.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:41.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:41.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:41.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:41.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:41.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:41.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:41.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:41.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:41.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:40 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:41.293 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:06:42.097 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T20:06:42.097 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 8s ago 27m - - 2023-10-01T20:06:42.097 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 6s ago 26m - - 2023-10-01T20:06:42.097 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (2m) 8s ago 23m - - 0.25.0 c8568f914cd2 ee46304105ca 2023-10-01T20:06:42.097 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (2m) 6s ago 23m 87.0M - 9.4.7 2c41d148cca3 e676c0004015 2023-10-01T20:06:42.097 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (3m) 8s ago 22m - - 3.5 e1d6a67b021e 7ef2818f7c68 2023-10-01T20:06:42.097 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283,8765 running (3m) 6s ago 26m 446M - 18.0.0-6505-g37d71fd3 bea6d6056140 b5797dd6a1b2 2023-10-01T20:06:42.097 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (12m) 8s ago 27m - - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T20:06:42.097 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 starting - - - 2048M 2023-10-01T20:06:42.098 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (75s) 6s ago 26m 57.8M 2048M 18.0.0-6505-g37d71fd3 bea6d6056140 805778222e17 2023-10-01T20:06:42.098 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (26m) 8s ago 26m - 2048M 17.2.0 e1d6a67b021e 965ad47ba362 2023-10-01T20:06:42.098 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (2m) 8s ago 23m - - 1.5.0 0da6a335fe13 175d4990d5fd 2023-10-01T20:06:42.098 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (2m) 6s ago 23m 18.2M - 1.5.0 0da6a335fe13 b2605eeb2dd4 2023-10-01T20:06:42.098 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (26m) 8s ago 26m - 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T20:06:42.098 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (25m) 8s ago 25m - 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T20:06:42.098 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (25m) 8s ago 25m - 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T20:06:42.098 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (25m) 8s ago 25m - 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T20:06:42.098 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (25m) 6s ago 25m 197M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T20:06:42.098 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (24m) 6s ago 24m 187M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T20:06:42.098 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (24m) 6s ago 24m 150M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T20:06:42.098 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (24m) 6s ago 24m 185M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T20:06:42.099 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (2m) 6s ago 23m 60.3M - 2.43.0 a07b618ecd1d 79b0216ed806 2023-10-01T20:06:42.099 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (22m) 8s ago 22m - - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T20:06:42.099 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (22m) 6s ago 22m 97.4M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T20:06:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='client.54102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:42.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:41 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='client.54102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:42.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='client.54102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:42.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:42.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:42.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:42.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:42.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:42.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:41 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:42.624 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:06:42.624 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2023-10-01T20:06:42.624 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2023-10-01T20:06:42.624 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:06:42.624 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:06:42.625 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2023-10-01T20:06:42.625 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:06:42.625 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:06:42.625 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2023-10-01T20:06:42.625 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2023-10-01T20:06:42.625 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:06:42.625 INFO:teuthology.orchestra.run.smithi057.stdout: "rgw": { 2023-10-01T20:06:42.626 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T20:06:42.626 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:06:42.626 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2023-10-01T20:06:42.626 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 11, 2023-10-01T20:06:42.626 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 4 2023-10-01T20:06:42.626 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T20:06:42.627 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:06:43.050 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:06:43.050 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24", 2023-10-01T20:06:43.050 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2023-10-01T20:06:43.051 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mon on host(s) smithi057", 2023-10-01T20:06:43.051 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2023-10-01T20:06:43.051 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "0/2 daemons upgraded", 2023-10-01T20:06:43.051 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mon daemons", 2023-10-01T20:06:43.051 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2023-10-01T20:06:43.051 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:06:43.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='client.34187 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:43.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.078 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.079 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.079 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.079 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.079 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.079 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.079 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.079 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.079 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.079 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='client.25600 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:43.079 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.079 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:43.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.081 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.081 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='client.34187 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:43.081 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.081 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.081 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.081 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.081 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.081 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.081 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.081 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.082 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.082 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.082 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.082 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.082 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.082 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.082 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.082 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='client.25600 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:43.082 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.082 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.082 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.083 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.083 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.083 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.083 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.083 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:43.083 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.083 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.083 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='client.? 172.21.15.57:0/2084463771' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.085 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.085 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='client.? 172.21.15.57:0/2084463771' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:42 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='client.34187 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:43.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='client.25600 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:43.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:43.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:43.138 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:43.139 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='client.? 172.21.15.57:0/2084463771' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.139 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.139 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:43.139 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:43.139 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:43.139 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:43.139 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:42 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.078 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.078 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.078 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.078 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.078 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.078 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.078 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.078 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:44.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.079 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.080 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.080 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.080 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.080 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.080 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.080 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.080 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.080 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.080 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.080 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.081 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.081 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.081 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.081 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.081 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.081 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.081 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.082 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.082 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.082 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.082 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.082 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.082 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.082 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.082 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.082 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:44.082 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.082 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.085 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.086 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.087 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:43 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:06:44.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.137 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.996 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:06:44] "GET /metrics HTTP/1.1" 200 37853 "" "Prometheus/2.43.0" 2023-10-01T20:06:44.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.999 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.999 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:44.999 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:44.999 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:44.999 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:44.999 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:44.999 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.999 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:44.999 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:44.999 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:45.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:45.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:45.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2023-10-01T20:06:45.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2023-10-01T20:06:45.001 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.001 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.001 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.001 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:45.001 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.001 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:45.001 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:45.001 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.002 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.002 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.002 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:45.002 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.002 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:45.002 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:45.002 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.002 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.002 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.003 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:45.003 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.003 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:45.003 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:45.003 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.003 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.003 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:45.003 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.003 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:45.003 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.003 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:45.004 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:45.004 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.004 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.004 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.004 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.004 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.004 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.004 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:45.004 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2023-10-01T20:06:45.004 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:44 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2023-10-01T20:06:45.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:45.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:45.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:45.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:45.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:45.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:45.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.135 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:45.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:45.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:45.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:45.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:45.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:06:45.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2023-10-01T20:06:45.136 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2023-10-01T20:06:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[205388]: Upgrade: It appears safe to stop mon.c 2023-10-01T20:06:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[205388]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:46.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[114331]: Upgrade: It appears safe to stop mon.c 2023-10-01T20:06:46.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:46.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:46.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:46.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[114331]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:46.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[114331]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:46.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:46.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:46.372 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:45 smithi057 ceph-mon[114331]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:45 smithi087 ceph-mon[178735]: Upgrade: It appears safe to stop mon.c 2023-10-01T20:06:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:45 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:45 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:45 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:45 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:45 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:47.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:47 smithi087 ceph-mon[178735]: Upgrade: Updating mon.c 2023-10-01T20:06:47.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:47 smithi087 ceph-mon[178735]: Deploying daemon mon.c on smithi057 2023-10-01T20:06:47.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:47 smithi087 ceph-mon[178735]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:47.389 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:47 smithi057 ceph-mon[205388]: Upgrade: Updating mon.c 2023-10-01T20:06:47.389 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:47 smithi057 ceph-mon[205388]: Deploying daemon mon.c on smithi057 2023-10-01T20:06:47.389 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:47 smithi057 ceph-mon[205388]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:47.390 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:47 smithi057 ceph-mon[114331]: Upgrade: Updating mon.c 2023-10-01T20:06:47.390 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:47 smithi057 ceph-mon[114331]: Deploying daemon mon.c on smithi057 2023-10-01T20:06:47.390 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:47 smithi057 ceph-mon[114331]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:47.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:47 smithi057 systemd[1]: Stopping Ceph mon.c for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:06:48.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:47 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-c[114308]: 2023-10-01T20:06:47.781+0000 7fdce7320700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:06:48.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:47 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-c[114308]: 2023-10-01T20:06:47.781+0000 7fdce7320700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-01T20:06:48.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:47 smithi057 bash[208871]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-c 2023-10-01T20:06:48.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.c.service: Succeeded. 2023-10-01T20:06:48.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 systemd[1]: Stopped Ceph mon.c for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:06:48.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 systemd[1]: Starting Ceph mon.c for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:06:49.108 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 bash[209022]: 8425f2f44a60465edc34309d8e14e4dc55ba2813b8a8f6ceda98408ea86cd73c 2023-10-01T20:06:49.108 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 systemd[1]: Started Ceph mon.c for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:06:49.108 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: set uid:gid to 167:167 (ceph:ceph) 2023-10-01T20:06:49.108 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev), process ceph-mon, pid 7 2023-10-01T20:06:49.108 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: pidfile_write: ignore empty --pid-file 2023-10-01T20:06:49.108 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: load: jerasure load: lrc 2023-10-01T20:06:49.109 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: RocksDB version: 7.9.2 2023-10-01T20:06:49.109 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Git sha 0 2023-10-01T20:06:49.109 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Compile date 2023-10-01 17:30:24 2023-10-01T20:06:49.109 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: DB SUMMARY 2023-10-01T20:06:49.109 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: DB Session ID: XDGGM9JI8XJQWKHNZ320 2023-10-01T20:06:49.109 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: CURRENT file: CURRENT 2023-10-01T20:06:49.109 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: IDENTITY file: IDENTITY 2023-10-01T20:06:49.109 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: MANIFEST file: MANIFEST-000009 size: 3802 Bytes 2023-10-01T20:06:49.109 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 1, files: 000069.sst 2023-10-01T20:06:49.109 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000067.log size: 5381062 ; 2023-10-01T20:06:49.110 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.error_if_exists: 0 2023-10-01T20:06:49.110 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.create_if_missing: 0 2023-10-01T20:06:49.110 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.paranoid_checks: 1 2023-10-01T20:06:49.110 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-01T20:06:49.110 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-01T20:06:49.110 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-01T20:06:49.110 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.env: 0x55dbe1104da0 2023-10-01T20:06:49.110 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.fs: PosixFileSystem 2023-10-01T20:06:49.110 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.info_log: 0x55dbe34055a0 2023-10-01T20:06:49.110 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_file_opening_threads: 16 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.statistics: (nil) 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.use_fsync: 0 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_log_file_size: 0 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.keep_log_file_num: 1000 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.recycle_log_file_num: 0 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.allow_fallocate: 1 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.allow_mmap_reads: 0 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.allow_mmap_writes: 0 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.use_direct_reads: 0 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-01T20:06:49.111 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.create_missing_column_families: 0 2023-10-01T20:06:49.113 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.db_log_dir: 2023-10-01T20:06:49.113 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.wal_dir: 2023-10-01T20:06:49.113 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-01T20:06:49.113 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-01T20:06:49.113 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-01T20:06:49.113 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-01T20:06:49.113 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-01T20:06:49.113 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-01T20:06:49.113 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.advise_random_on_open: 1 2023-10-01T20:06:49.113 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.db_write_buffer_size: 0 2023-10-01T20:06:49.114 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.write_buffer_manager: 0x55dbe33bd360 2023-10-01T20:06:49.114 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-01T20:06:49.114 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-01T20:06:49.114 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-01T20:06:49.114 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.rate_limiter: (nil) 2023-10-01T20:06:49.114 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-01T20:06:49.114 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.wal_recovery_mode: 2 2023-10-01T20:06:49.114 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.enable_thread_tracking: 0 2023-10-01T20:06:49.114 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.enable_pipelined_write: 0 2023-10-01T20:06:49.114 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.unordered_write: 0 2023-10-01T20:06:49.114 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-01T20:06:49.114 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.row_cache: None 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.wal_filter: None 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.allow_ingest_behind: 0 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.two_write_queues: 0 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.manual_wal_flush: 0 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.wal_compression: 0 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.atomic_flush: 0 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-01T20:06:49.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.log_readahead_size: 0 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.best_efforts_recovery: 0 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.allow_data_in_errors: 0 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.db_host_id: __hostname__ 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_background_jobs: 2 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_background_compactions: -1 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_subcompactions: 1 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-01T20:06:49.116 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-01T20:06:49.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-01T20:06:49.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_total_wal_size: 0 2023-10-01T20:06:49.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-01T20:06:49.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-01T20:06:49.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-01T20:06:49.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-01T20:06:49.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_open_files: -1 2023-10-01T20:06:49.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bytes_per_sync: 0 2023-10-01T20:06:49.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-01T20:06:49.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-01T20:06:49.117 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_readahead_size: 0 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_background_flushes: -1 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Compression algorithms supported: 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: kZSTD supported: 0 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: kXpressCompression supported: 0 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: kLZ4HCCompression supported: 1 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: kZlibCompression supported: 1 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: kSnappyCompression supported: 1 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: kLZ4Compression supported: 1 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: kBZip2Compression supported: 0 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-01T20:06:49.118 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000009 2023-10-01T20:06:49.119 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-01T20:06:49.119 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-01T20:06:49.119 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.merge_operator: 2023-10-01T20:06:49.119 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_filter: None 2023-10-01T20:06:49.119 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_filter_factory: None 2023-10-01T20:06:49.119 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.sst_partitioner_factory: None 2023-10-01T20:06:49.119 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-01T20:06:49.120 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-01T20:06:49.120 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55dbe34056a0) 2023-10-01T20:06:49.121 INFO:journalctl@ceph.mon.c.smithi057.stdout: cache_index_and_filter_blocks: 1 2023-10-01T20:06:49.121 INFO:journalctl@ceph.mon.c.smithi057.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-01T20:06:49.121 INFO:journalctl@ceph.mon.c.smithi057.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-01T20:06:49.121 INFO:journalctl@ceph.mon.c.smithi057.stdout: pin_top_level_index_and_filter: 1 2023-10-01T20:06:49.122 INFO:journalctl@ceph.mon.c.smithi057.stdout: index_type: 0 2023-10-01T20:06:49.122 INFO:journalctl@ceph.mon.c.smithi057.stdout: data_block_index_type: 0 2023-10-01T20:06:49.122 INFO:journalctl@ceph.mon.c.smithi057.stdout: index_shortening: 1 2023-10-01T20:06:49.122 INFO:journalctl@ceph.mon.c.smithi057.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-01T20:06:49.122 INFO:journalctl@ceph.mon.c.smithi057.stdout: checksum: 4 2023-10-01T20:06:49.122 INFO:journalctl@ceph.mon.c.smithi057.stdout: no_block_cache: 0 2023-10-01T20:06:49.122 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_cache: 0x55dbe345b1f0 2023-10-01T20:06:49.122 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_cache_name: BinnedLRUCache 2023-10-01T20:06:49.122 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_cache_options: 2023-10-01T20:06:49.122 INFO:journalctl@ceph.mon.c.smithi057.stdout: capacity : 536870912 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: num_shard_bits : 4 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: strict_capacity_limit : 0 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: high_pri_pool_ratio: 0.000 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_cache_compressed: (nil) 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: persistent_cache: (nil) 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_size: 4096 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_size_deviation: 10 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_restart_interval: 16 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: index_block_restart_interval: 1 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: metadata_block_size: 4096 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: partition_filters: 0 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: use_delta_encoding: 1 2023-10-01T20:06:49.123 INFO:journalctl@ceph.mon.c.smithi057.stdout: filter_policy: bloomfilter 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout: whole_key_filtering: 1 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout: verify_compression: 0 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout: read_amp_bytes_per_bit: 0 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout: format_version: 5 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout: enable_index_compression: 1 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout: block_align: 0 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout: max_auto_readahead_size: 262144 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout: prepopulate_block_cache: 0 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout: initial_auto_readahead_size: 8192 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout: num_file_reads_for_auto_readahead: 2 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.write_buffer_size: 33554432 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_write_buffer_number: 2 2023-10-01T20:06:49.124 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compression: NoCompression 2023-10-01T20:06:49.125 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bottommost_compression: Disabled 2023-10-01T20:06:49.125 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.prefix_extractor: nullptr 2023-10-01T20:06:49.125 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-01T20:06:49.125 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.num_levels: 7 2023-10-01T20:06:49.125 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-01T20:06:49.125 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-01T20:06:49.125 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-01T20:06:49.125 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-01T20:06:49.125 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-01T20:06:49.125 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-01T20:06:49.125 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-01T20:06:49.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-01T20:06:49.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-01T20:06:49.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-01T20:06:49.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-01T20:06:49.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-01T20:06:49.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-01T20:06:49.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compression_opts.level: 32767 2023-10-01T20:06:49.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compression_opts.strategy: 0 2023-10-01T20:06:49.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-01T20:06:49.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-01T20:06:49.126 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compression_opts.enabled: false 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.target_file_size_base: 67108864 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-01T20:06:49.127 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-01T20:06:49.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-01T20:06:49.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-01T20:06:49.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-01T20:06:49.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-01T20:06:49.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-01T20:06:49.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-01T20:06:49.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-01T20:06:49.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.arena_block_size: 1048576 2023-10-01T20:06:49.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-01T20:06:49.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-01T20:06:49.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.disable_auto_compactions: 0 2023-10-01T20:06:49.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.table_properties_collectors: 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.inplace_update_support: 0 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-01T20:06:49.129 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-01T20:06:49.130 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-01T20:06:49.130 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-01T20:06:49.130 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.bloom_locality: 0 2023-10-01T20:06:49.130 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.max_successive_merges: 0 2023-10-01T20:06:49.130 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-01T20:06:49.130 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.paranoid_file_checks: 0 2023-10-01T20:06:49.130 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.force_consistency_checks: 1 2023-10-01T20:06:49.130 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.report_bg_io_stats: 0 2023-10-01T20:06:49.130 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.ttl: 2592000 2023-10-01T20:06:49.130 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-01T20:06:49.130 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-01T20:06:49.130 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.enable_blob_files: false 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.min_blob_size: 0 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.blob_file_size: 268435456 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.blob_file_starting_level: 0 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 69.sst 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2023-10-01T20:06:49.131 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 71, last_sequence is 38825, log_number is 67,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-01T20:06:49.132 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 67 2023-10-01T20:06:49.132 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 286bb91b-5d47-4914-8d12-4e985d44fd57 2023-10-01T20:06:49.132 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696190808852546, "job": 1, "event": "recovery_started", "wal_files": [67]} 2023-10-01T20:06:49.132 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #67 mode 2 2023-10-01T20:06:49.132 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696190808867133, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 72, "file_size": 3299373, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 38831, "largest_seqno": 40903, "table_properties": {"data_size": 3290929, "index_size": 4928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2501, "raw_key_size": 23592, "raw_average_key_size": 23, "raw_value_size": 3272142, "raw_average_value_size": 3325, "num_data_blocks": 217, "num_entries": 984, "num_filter_entries": 984, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696190808, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "286bb91b-5d47-4914-8d12-4e985d44fd57", "db_session_id": "XDGGM9JI8XJQWKHNZ320", "orig_file_number": 72, "seqno_to_time_mapping": "N/A"}} 2023-10-01T20:06:49.132 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696190808867238, "job": 1, "event": "recovery_finished"} 2023-10-01T20:06:49.132 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [db/version_set.cc:5047] Creating manifest 74 2023-10-01T20:06:49.132 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2023-10-01T20:06:49.132 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000067.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-01T20:06:49.132 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55dbe34f8000 2023-10-01T20:06:49.132 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: DB pointer 0x55dbe34e4000 2023-10-01T20:06:49.132 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout: ** DB Stats ** 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-01T20:06:49.133 INFO:journalctl@ceph.mon.c.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-01T20:06:49.134 INFO:journalctl@ceph.mon.c.smithi057.stdout: L0 1/0 3.15 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 583.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2023-10-01T20:06:49.134 INFO:journalctl@ceph.mon.c.smithi057.stdout: L6 1/0 7.95 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2023-10-01T20:06:49.134 INFO:journalctl@ceph.mon.c.smithi057.stdout: Sum 2/0 11.09 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 583.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2023-10-01T20:06:49.134 INFO:journalctl@ceph.mon.c.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 583.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2023-10-01T20:06:49.134 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T20:06:49.134 INFO:journalctl@ceph.mon.c.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-01T20:06:49.134 INFO:journalctl@ceph.mon.c.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-01T20:06:49.134 INFO:journalctl@ceph.mon.c.smithi057.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-01T20:06:49.134 INFO:journalctl@ceph.mon.c.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 583.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2023-10-01T20:06:49.134 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T20:06:49.134 INFO:journalctl@ceph.mon.c.smithi057.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-01T20:06:49.134 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T20:06:49.135 INFO:journalctl@ceph.mon.c.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-01T20:06:49.135 INFO:journalctl@ceph.mon.c.smithi057.stdout: Flush(GB): cumulative 0.003, interval 0.003 2023-10-01T20:06:49.135 INFO:journalctl@ceph.mon.c.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-01T20:06:49.135 INFO:journalctl@ceph.mon.c.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-01T20:06:49.135 INFO:journalctl@ceph.mon.c.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-01T20:06:49.135 INFO:journalctl@ceph.mon.c.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-01T20:06:49.135 INFO:journalctl@ceph.mon.c.smithi057.stdout: Cumulative compaction: 0.00 GB write, 176.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T20:06:49.135 INFO:journalctl@ceph.mon.c.smithi057.stdout: Interval compaction: 0.00 GB write, 176.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-01T20:06:49.135 INFO:journalctl@ceph.mon.c.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-01T20:06:49.135 INFO:journalctl@ceph.mon.c.smithi057.stdout: Block cache BinnedLRUCache@0x55dbe345b1f0#7 capacity: 512.00 MB usage: 7.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2023-10-01T20:06:49.135 INFO:journalctl@ceph.mon.c.smithi057.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.53 KB,0.000482798%) IndexBlock(1,5.08 KB,0.000968575%) Misc(1,0.00 KB,0%) 2023-10-01T20:06:49.135 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: starting mon.c rank 1 at public addrs [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] at bind addrs [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: mon.c@-1(???) e3 preinit fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: mon.c@-1(???).mds e1 new map 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: mon.c@-1(???).mds e1 print_map 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout: e1 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout: legacy client fscid: -1 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout: 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout: No filesystems configured 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: mon.c@-1(???).osd e108 crush map has features 3314933000854323200, adjusting msgr requires 2023-10-01T20:06:49.136 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: mon.c@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2023-10-01T20:06:49.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: mon.c@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2023-10-01T20:06:49.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: mon.c@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2023-10-01T20:06:49.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: mon.c@-1(???).paxosservice(auth 1..29) refresh upgraded, format 0 -> 3 2023-10-01T20:06:49.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: expand_channel_meta expand map: {default=false} 2023-10-01T20:06:49.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: expand_channel_meta from 'false' to 'false' 2023-10-01T20:06:49.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: expand_channel_meta expanded map: {default=false} 2023-10-01T20:06:49.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: expand_channel_meta expand map: {default=info} 2023-10-01T20:06:49.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: expand_channel_meta from 'info' to 'info' 2023-10-01T20:06:49.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: expand_channel_meta expanded map: {default=info} 2023-10-01T20:06:49.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: expand_channel_meta expand map: {default=daemon} 2023-10-01T20:06:49.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: expand_channel_meta from 'daemon' to 'daemon' 2023-10-01T20:06:49.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: expand_channel_meta expanded map: {default=daemon} 2023-10-01T20:06:49.138 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: expand_channel_meta expand map: {default=debug} 2023-10-01T20:06:49.138 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: expand_channel_meta from 'debug' to 'debug' 2023-10-01T20:06:49.138 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:48 smithi057 ceph-mon[209060]: expand_channel_meta expanded map: {default=debug} 2023-10-01T20:06:54.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:53 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:06:53.879+0000 7fa4cc645700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-01T20:06:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[205388]: overall HEALTH_OK 2023-10-01T20:06:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[205388]: mon.a calling monitor election 2023-10-01T20:06:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[205388]: mon.c calling monitor election 2023-10-01T20:06:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[205388]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-01T20:06:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[205388]: monmap e4: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} removed_ranks: {} 2023-10-01T20:06:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[205388]: fsmap 2023-10-01T20:06:54.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[205388]: osdmap e108: 8 total, 8 up, 8 in 2023-10-01T20:06:54.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[205388]: mgrmap e50: y(active, since 29s), standbys: x 2023-10-01T20:06:54.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[205388]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-01T20:06:54.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[205388]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-01T20:06:54.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[205388]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-01T20:06:54.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[205388]: mon.b (rank 2) addr [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] is down (out of quorum) 2023-10-01T20:06:54.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[209060]: overall HEALTH_OK 2023-10-01T20:06:54.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[209060]: mon.a calling monitor election 2023-10-01T20:06:54.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[209060]: mon.c calling monitor election 2023-10-01T20:06:54.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[209060]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-01T20:06:54.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[209060]: monmap e4: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} removed_ranks: {} 2023-10-01T20:06:54.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[209060]: fsmap 2023-10-01T20:06:54.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[209060]: osdmap e108: 8 total, 8 up, 8 in 2023-10-01T20:06:54.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[209060]: mgrmap e50: y(active, since 29s), standbys: x 2023-10-01T20:06:54.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[209060]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-01T20:06:54.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[209060]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-01T20:06:54.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[209060]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-01T20:06:54.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-mon[209060]: mon.b (rank 2) addr [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] is down (out of quorum) 2023-10-01T20:06:55.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:06:54 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:06:54] "GET /metrics HTTP/1.1" 200 37853 "" "Prometheus/2.43.0" 2023-10-01T20:06:55.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: overall HEALTH_OK 2023-10-01T20:06:55.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: mon.a calling monitor election 2023-10-01T20:06:55.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: mon.c calling monitor election 2023-10-01T20:06:55.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-01T20:06:55.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: monmap e4: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} removed_ranks: {} 2023-10-01T20:06:55.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: fsmap 2023-10-01T20:06:55.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: osdmap e108: 8 total, 8 up, 8 in 2023-10-01T20:06:55.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: mgrmap e50: y(active, since 29s), standbys: x 2023-10-01T20:06:55.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-01T20:06:55.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-01T20:06:55.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-01T20:06:55.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: mon.b (rank 2) addr [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] is down (out of quorum) 2023-10-01T20:06:55.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:06:55.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:06:55.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:06:55.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: mon.b calling monitor election 2023-10-01T20:06:55.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: mon.c calling monitor election 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: mon.a calling monitor election 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: monmap e4: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} removed_ranks: {} 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: fsmap 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: osdmap e108: 8 total, 8 up, 8 in 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: mgrmap e50: y(active, since 31s), standbys: x 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-01T20:06:55.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: Cluster is now healthy 2023-10-01T20:06:55.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: overall HEALTH_OK 2023-10-01T20:06:55.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='' 2023-10-01T20:06:55.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:55 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:56.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:06:56.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:06:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:06:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: mon.b calling monitor election 2023-10-01T20:06:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:06:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: mon.c calling monitor election 2023-10-01T20:06:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: mon.a calling monitor election 2023-10-01T20:06:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T20:06:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: monmap e4: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} removed_ranks: {} 2023-10-01T20:06:56.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: fsmap 2023-10-01T20:06:56.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: osdmap e108: 8 total, 8 up, 8 in 2023-10-01T20:06:56.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: mgrmap e50: y(active, since 31s), standbys: x 2023-10-01T20:06:56.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-01T20:06:56.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: Cluster is now healthy 2023-10-01T20:06:56.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: overall HEALTH_OK 2023-10-01T20:06:56.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='' 2023-10-01T20:06:56.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:56.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-01T20:06:56.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-01T20:06:56.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-01T20:06:56.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: mon.b calling monitor election 2023-10-01T20:06:56.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:56.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:56.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:56.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:06:56.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:06:56.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: mon.c calling monitor election 2023-10-01T20:06:56.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: mon.a calling monitor election 2023-10-01T20:06:56.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-01T20:06:56.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: monmap e4: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0]} removed_ranks: {} 2023-10-01T20:06:56.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: fsmap 2023-10-01T20:06:56.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: osdmap e108: 8 total, 8 up, 8 in 2023-10-01T20:06:56.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: mgrmap e50: y(active, since 31s), standbys: x 2023-10-01T20:06:56.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-01T20:06:56.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: Cluster is now healthy 2023-10-01T20:06:56.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: overall HEALTH_OK 2023-10-01T20:06:56.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='' 2023-10-01T20:06:56.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:55 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:56.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[205388]: Reconfiguring mon.a (monmap changed)... 2023-10-01T20:06:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:56.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[205388]: Reconfiguring daemon mon.a on smithi057 2023-10-01T20:06:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[205388]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:56.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:56.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:56.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:56.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:56.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:56.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[209060]: Reconfiguring mon.a (monmap changed)... 2023-10-01T20:06:56.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:56.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:56.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:56.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[209060]: Reconfiguring daemon mon.a on smithi057 2023-10-01T20:06:56.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:56 smithi057 ceph-mon[209060]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:57.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:06:57.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:57.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:06:57.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:56 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:57.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:56 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:06:57.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:56 smithi087 ceph-mon[178735]: Reconfiguring mon.a (monmap changed)... 2023-10-01T20:06:57.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:57.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:57.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:57.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:56 smithi087 ceph-mon[178735]: Reconfiguring daemon mon.a on smithi057 2023-10-01T20:06:57.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:56 smithi087 ceph-mon[178735]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:06:57.921 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:57.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:57.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[209060]: Reconfiguring mgr.y (monmap changed)... 2023-10-01T20:06:57.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:06:57.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:06:57.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T20:06:57.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:57.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[209060]: Reconfiguring daemon mgr.y on smithi057 2023-10-01T20:06:57.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:57.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[205388]: Reconfiguring mgr.y (monmap changed)... 2023-10-01T20:06:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:06:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:06:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T20:06:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:57 smithi057 ceph-mon[205388]: Reconfiguring daemon mgr.y on smithi057 2023-10-01T20:06:58.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:57 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:58.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:57 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:58.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:57 smithi087 ceph-mon[178735]: Reconfiguring mgr.y (monmap changed)... 2023-10-01T20:06:58.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:06:58.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:57 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:06:58.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T20:06:58.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:58.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:57 smithi087 ceph-mon[178735]: Reconfiguring daemon mgr.y on smithi057 2023-10-01T20:06:59.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[209060]: Reconfiguring mon.c (monmap changed)... 2023-10-01T20:06:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[209060]: Reconfiguring daemon mon.c on smithi057 2023-10-01T20:06:59.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[209060]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[205388]: Reconfiguring mon.c (monmap changed)... 2023-10-01T20:06:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[205388]: Reconfiguring daemon mon.c on smithi057 2023-10-01T20:06:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:06:58 smithi057 ceph-mon[205388]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:06:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:58 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:58 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:58 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:58 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:58 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:06:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:58 smithi087 ceph-mon[178735]: Reconfiguring mon.c (monmap changed)... 2023-10-01T20:06:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:06:59.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:06:59.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:06:59.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:58 smithi087 ceph-mon[178735]: Reconfiguring daemon mon.c on smithi057 2023-10-01T20:06:59.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:06:58 smithi087 ceph-mon[178735]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:00.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:00 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:00.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:00 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:00.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:00 smithi057 ceph-mon[205388]: Reconfiguring osd.0 (monmap changed)... 2023-10-01T20:07:00.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-01T20:07:00.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:00.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:00 smithi057 ceph-mon[205388]: Reconfiguring daemon osd.0 on smithi057 2023-10-01T20:07:00.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:00 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:00.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:00 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:00.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:00 smithi057 ceph-mon[209060]: Reconfiguring osd.0 (monmap changed)... 2023-10-01T20:07:00.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-01T20:07:00.172 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:00.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:00 smithi057 ceph-mon[209060]: Reconfiguring daemon osd.0 on smithi057 2023-10-01T20:07:00.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:00 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:00.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:00 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:00.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:00 smithi087 ceph-mon[178735]: Reconfiguring osd.0 (monmap changed)... 2023-10-01T20:07:00.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-01T20:07:00.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:00.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:00 smithi087 ceph-mon[178735]: Reconfiguring daemon osd.0 on smithi057 2023-10-01T20:07:01.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[205388]: Reconfiguring osd.1 (monmap changed)... 2023-10-01T20:07:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-01T20:07:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[205388]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:01.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[205388]: Reconfiguring daemon osd.1 on smithi057 2023-10-01T20:07:01.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:01.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:01.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[209060]: Reconfiguring osd.1 (monmap changed)... 2023-10-01T20:07:01.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-01T20:07:01.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:01.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[209060]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:01.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:01 smithi057 ceph-mon[209060]: Reconfiguring daemon osd.1 on smithi057 2023-10-01T20:07:01.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:01 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:01.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:01 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:01.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:01 smithi087 ceph-mon[178735]: Reconfiguring osd.1 (monmap changed)... 2023-10-01T20:07:01.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-01T20:07:01.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:01.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:01 smithi087 ceph-mon[178735]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:01.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:01 smithi087 ceph-mon[178735]: Reconfiguring daemon osd.1 on smithi057 2023-10-01T20:07:02.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:02.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:02.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:02 smithi087 ceph-mon[178735]: Reconfiguring osd.2 (monmap changed)... 2023-10-01T20:07:02.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-01T20:07:02.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:02.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:02 smithi087 ceph-mon[178735]: Reconfiguring daemon osd.2 on smithi057 2023-10-01T20:07:02.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:02.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:02.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:02 smithi057 ceph-mon[209060]: Reconfiguring osd.2 (monmap changed)... 2023-10-01T20:07:02.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-01T20:07:02.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:02.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:02 smithi057 ceph-mon[209060]: Reconfiguring daemon osd.2 on smithi057 2023-10-01T20:07:02.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:02.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:02.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:02 smithi057 ceph-mon[205388]: Reconfiguring osd.2 (monmap changed)... 2023-10-01T20:07:02.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-01T20:07:02.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:02.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:02 smithi057 ceph-mon[205388]: Reconfiguring daemon osd.2 on smithi057 2023-10-01T20:07:03.421 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[209060]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:03.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:03.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:03.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[209060]: Reconfiguring osd.3 (monmap changed)... 2023-10-01T20:07:03.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-01T20:07:03.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:03.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[209060]: Reconfiguring daemon osd.3 on smithi057 2023-10-01T20:07:03.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[205388]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:03.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:03.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:03.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[205388]: Reconfiguring osd.3 (monmap changed)... 2023-10-01T20:07:03.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-01T20:07:03.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:03.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:03 smithi057 ceph-mon[205388]: Reconfiguring daemon osd.3 on smithi057 2023-10-01T20:07:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:03 smithi087 ceph-mon[178735]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:03 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:03 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:03 smithi087 ceph-mon[178735]: Reconfiguring osd.3 (monmap changed)... 2023-10-01T20:07:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:03 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-01T20:07:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:03 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:03.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:03 smithi087 ceph-mon[178735]: Reconfiguring daemon osd.3 on smithi057 2023-10-01T20:07:04.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[205388]: Reconfiguring rgw.foo.smithi057.dbidoa (monmap changed)... 2023-10-01T20:07:04.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi057.dbidoa", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T20:07:04.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi057.dbidoa", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T20:07:04.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:04.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[205388]: Reconfiguring daemon rgw.foo.smithi057.dbidoa on smithi057 2023-10-01T20:07:04.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.694 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.694 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.694 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[209060]: Reconfiguring rgw.foo.smithi057.dbidoa (monmap changed)... 2023-10-01T20:07:04.694 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi057.dbidoa", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T20:07:04.694 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi057.dbidoa", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T20:07:04.694 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:04.695 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[209060]: Reconfiguring daemon rgw.foo.smithi057.dbidoa on smithi057 2023-10-01T20:07:04.695 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.695 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.695 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:04 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:04 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:04 smithi087 ceph-mon[178735]: Reconfiguring rgw.foo.smithi057.dbidoa (monmap changed)... 2023-10-01T20:07:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:04 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi057.dbidoa", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T20:07:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:04 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi057.dbidoa", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T20:07:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:04 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:04 smithi087 ceph-mon[178735]: Reconfiguring daemon rgw.foo.smithi057.dbidoa on smithi057 2023-10-01T20:07:04.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:04 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:04 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:04.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:04 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:05.171 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:07:04 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:07:04] "GET /metrics HTTP/1.1" 200 37926 "" "Prometheus/2.43.0" 2023-10-01T20:07:05.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:05 smithi087 ceph-mon[178735]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:05.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:05 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:05.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:05 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:05.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:05 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:07:05.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:05 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:07:05.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:05 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:05.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:05 smithi057 ceph-mon[209060]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:05.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:05 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:05.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:05 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:05.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:05 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:07:05.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:05 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:07:05.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:05 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:05.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:05 smithi057 ceph-mon[205388]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:05.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:05 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:05.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:05 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:05.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:05 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-01T20:07:05.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:05 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-01T20:07:05.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:05 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:06 smithi087 ceph-mon[178735]: Reconfiguring mon.b (monmap changed)... 2023-10-01T20:07:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:06 smithi087 ceph-mon[178735]: Reconfiguring daemon mon.b on smithi087 2023-10-01T20:07:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:06 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:06 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:06 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:06 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:07:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:06 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:07:06.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:06 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T20:07:06.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:06 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:06.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[205388]: Reconfiguring mon.b (monmap changed)... 2023-10-01T20:07:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[205388]: Reconfiguring daemon mon.b on smithi087 2023-10-01T20:07:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[205388]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:07:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:07:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T20:07:06.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[209060]: Reconfiguring mon.b (monmap changed)... 2023-10-01T20:07:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[209060]: Reconfiguring daemon mon.b on smithi087 2023-10-01T20:07:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[209060]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:07:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-01T20:07:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-01T20:07:06.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:06 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:07.541 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: Reconfiguring mgr.x (monmap changed)... 2023-10-01T20:07:07.541 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: Reconfiguring daemon mgr.x on smithi087 2023-10-01T20:07:07.541 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:07.541 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: Reconfiguring osd.4 (monmap changed)... 2023-10-01T20:07:07.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-01T20:07:07.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:07.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: Reconfiguring daemon osd.4 on smithi087 2023-10-01T20:07:07.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.542 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-01T20:07:07.543 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: Reconfiguring mgr.x (monmap changed)... 2023-10-01T20:07:07.543 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: Reconfiguring daemon mgr.x on smithi087 2023-10-01T20:07:07.543 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:07.543 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.543 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.543 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: Reconfiguring osd.4 (monmap changed)... 2023-10-01T20:07:07.543 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-01T20:07:07.543 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:07.543 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: Reconfiguring daemon osd.4 on smithi087 2023-10-01T20:07:07.543 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.543 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.544 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.544 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.544 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.544 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:07 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-01T20:07:07.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: Reconfiguring mgr.x (monmap changed)... 2023-10-01T20:07:07.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: Reconfiguring daemon mgr.x on smithi087 2023-10-01T20:07:07.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:07.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: Reconfiguring osd.4 (monmap changed)... 2023-10-01T20:07:07.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-01T20:07:07.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:07.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: Reconfiguring daemon osd.4 on smithi087 2023-10-01T20:07:07.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:07.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:07 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-01T20:07:08.825 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:08 smithi087 ceph-mon[178735]: Reconfiguring osd.5 (monmap changed)... 2023-10-01T20:07:08.825 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:08 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:08.825 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:08 smithi087 ceph-mon[178735]: Reconfiguring daemon osd.5 on smithi087 2023-10-01T20:07:08.825 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:08 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:08.825 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:08 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:08.826 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:08 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-01T20:07:08.826 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:08 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:08.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[205388]: Reconfiguring osd.5 (monmap changed)... 2023-10-01T20:07:08.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:08.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[205388]: Reconfiguring daemon osd.5 on smithi087 2023-10-01T20:07:08.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:08.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:08.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-01T20:07:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:08.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[209060]: Reconfiguring osd.5 (monmap changed)... 2023-10-01T20:07:08.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:08.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[209060]: Reconfiguring daemon osd.5 on smithi087 2023-10-01T20:07:08.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:08.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:08.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-01T20:07:08.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:08 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:09.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:09 smithi087 ceph-mon[178735]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:09.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:09 smithi087 ceph-mon[178735]: Reconfiguring osd.6 (monmap changed)... 2023-10-01T20:07:09.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:09 smithi087 ceph-mon[178735]: Reconfiguring daemon osd.6 on smithi087 2023-10-01T20:07:09.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:09 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:07:09.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:09 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:09.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:09 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:09.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:09 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-01T20:07:09.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:09 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:09.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[205388]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:09.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[205388]: Reconfiguring osd.6 (monmap changed)... 2023-10-01T20:07:09.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[205388]: Reconfiguring daemon osd.6 on smithi087 2023-10-01T20:07:09.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:07:09.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:09.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:09.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-01T20:07:09.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:09.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[209060]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:09.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[209060]: Reconfiguring osd.6 (monmap changed)... 2023-10-01T20:07:09.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[209060]: Reconfiguring daemon osd.6 on smithi087 2023-10-01T20:07:09.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:07:09.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:09.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:09.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-01T20:07:09.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:09 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:10.817 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:10 smithi087 ceph-mon[178735]: Reconfiguring osd.7 (monmap changed)... 2023-10-01T20:07:10.817 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:10 smithi087 ceph-mon[178735]: Reconfiguring daemon osd.7 on smithi087 2023-10-01T20:07:10.817 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:10 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:10.817 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:10 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:10.817 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:10 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi087.rxeurp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T20:07:10.817 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:10 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi087.rxeurp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T20:07:10.817 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:10 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[205388]: Reconfiguring osd.7 (monmap changed)... 2023-10-01T20:07:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[205388]: Reconfiguring daemon osd.7 on smithi087 2023-10-01T20:07:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi087.rxeurp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T20:07:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi087.rxeurp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T20:07:10.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:10.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[209060]: Reconfiguring osd.7 (monmap changed)... 2023-10-01T20:07:10.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[209060]: Reconfiguring daemon osd.7 on smithi087 2023-10-01T20:07:10.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:10.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:10.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi087.rxeurp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T20:07:10.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi087.rxeurp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-10-01T20:07:10.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:10 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:11.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: Reconfiguring rgw.foo.smithi087.rxeurp (monmap changed)... 2023-10-01T20:07:11.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: Reconfiguring daemon rgw.foo.smithi087.rxeurp on smithi087 2023-10-01T20:07:11.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:11.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2023-10-01T20:07:11.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2023-10-01T20:07:11.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:07:11.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:07:11.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2023-10-01T20:07:11.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:11.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:11.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2023-10-01T20:07:11.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:07:11.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:07:11.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2023-10-01T20:07:11.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:07:11.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:07:11.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:07:11.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:07:11.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:11 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2023-10-01T20:07:11.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:11.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: Reconfiguring rgw.foo.smithi087.rxeurp (monmap changed)... 2023-10-01T20:07:11.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: Reconfiguring daemon rgw.foo.smithi087.rxeurp on smithi087 2023-10-01T20:07:11.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:11.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2023-10-01T20:07:11.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2023-10-01T20:07:11.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:11.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: Reconfiguring rgw.foo.smithi087.rxeurp (monmap changed)... 2023-10-01T20:07:11.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: Reconfiguring daemon rgw.foo.smithi087.rxeurp on smithi087 2023-10-01T20:07:11.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:11.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2023-10-01T20:07:11.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2023-10-01T20:07:11.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.928 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.930 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:11.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:07:11.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:07:11.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2023-10-01T20:07:11.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:11.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:11.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2023-10-01T20:07:11.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:07:11.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:07:11.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2023-10-01T20:07:11.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:07:11.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:07:11.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:07:11.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:07:11.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2023-10-01T20:07:11.935 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.935 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.935 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:11.935 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.935 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2023-10-01T20:07:11.935 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.935 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2023-10-01T20:07:11.935 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:07:11.935 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:07:11.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2023-10-01T20:07:11.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:11.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:11.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2023-10-01T20:07:11.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:07:11.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:07:11.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2023-10-01T20:07:11.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:07:11.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:07:11.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:07:11.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:07:11.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:11 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2023-10-01T20:07:12.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all mon 2023-10-01T20:07:12.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all crash 2023-10-01T20:07:12.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all mds 2023-10-01T20:07:12.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all rbd-mirror 2023-10-01T20:07:12.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all cephfs-mirror 2023-10-01T20:07:12.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all ceph-exporter 2023-10-01T20:07:12.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all nfs 2023-10-01T20:07:12.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all nvmeof 2023-10-01T20:07:12.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all node-exporter 2023-10-01T20:07:12.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all prometheus 2023-10-01T20:07:12.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all alertmanager 2023-10-01T20:07:12.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all grafana 2023-10-01T20:07:12.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all loki 2023-10-01T20:07:12.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all promtail 2023-10-01T20:07:12.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: Upgrade: Finalizing container_image settings 2023-10-01T20:07:12.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:07:12.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:07:12.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:07:12.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:07:12.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2023-10-01T20:07:12.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:07:12.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:07:12.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2023-10-01T20:07:12.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:07:12.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:07:12.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:07:12.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:07:12.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2023-10-01T20:07:12.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:07:12.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:07:12.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2023-10-01T20:07:12.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:12.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:12.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:12.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:12.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:12.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:12 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:12.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all mon 2023-10-01T20:07:12.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all crash 2023-10-01T20:07:12.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all mds 2023-10-01T20:07:12.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all rbd-mirror 2023-10-01T20:07:12.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all cephfs-mirror 2023-10-01T20:07:12.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all ceph-exporter 2023-10-01T20:07:12.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all nfs 2023-10-01T20:07:12.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all nvmeof 2023-10-01T20:07:12.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all node-exporter 2023-10-01T20:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all prometheus 2023-10-01T20:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all alertmanager 2023-10-01T20:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all grafana 2023-10-01T20:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all loki 2023-10-01T20:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all promtail 2023-10-01T20:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: Upgrade: Finalizing container_image settings 2023-10-01T20:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2023-10-01T20:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:07:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:07:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2023-10-01T20:07:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:07:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:07:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:07:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:07:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2023-10-01T20:07:12.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all mon 2023-10-01T20:07:12.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all crash 2023-10-01T20:07:12.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all mds 2023-10-01T20:07:12.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all rbd-mirror 2023-10-01T20:07:12.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all cephfs-mirror 2023-10-01T20:07:12.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all ceph-exporter 2023-10-01T20:07:12.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all nfs 2023-10-01T20:07:12.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all nvmeof 2023-10-01T20:07:12.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all node-exporter 2023-10-01T20:07:12.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all prometheus 2023-10-01T20:07:12.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all alertmanager 2023-10-01T20:07:12.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all grafana 2023-10-01T20:07:12.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all loki 2023-10-01T20:07:12.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all promtail 2023-10-01T20:07:12.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: Upgrade: Finalizing container_image settings 2023-10-01T20:07:12.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:07:12.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:07:12.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:07:12.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:07:12.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2023-10-01T20:07:12.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:07:12.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:07:12.926 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2023-10-01T20:07:12.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:07:12.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:07:12.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:07:12.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:07:12.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2023-10-01T20:07:12.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:07:12.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:07:12.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2023-10-01T20:07:12.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:07:12.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:07:12.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2023-10-01T20:07:12.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:12.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:12.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:12.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:12.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:12.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:12.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:12.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:12.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:07:12.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:07:12.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2023-10-01T20:07:12.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:07:12.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:07:12.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:07:12.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2023-10-01T20:07:12.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:12.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:12.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:12.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:12.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:12.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:12.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:12.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:12 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:13.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:13 smithi057 ceph-mon[205388]: Upgrade: Complete! 2023-10-01T20:07:13.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:13 smithi057 ceph-mon[205388]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:13.857 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:13 smithi057 ceph-mon[209060]: Upgrade: Complete! 2023-10-01T20:07:13.858 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:13 smithi057 ceph-mon[209060]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:13.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:13 smithi087 ceph-mon[178735]: Upgrade: Complete! 2023-10-01T20:07:13.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:13 smithi087 ceph-mon[178735]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:13.886 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch ps' 2023-10-01T20:07:14.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:14 smithi087 ceph-mon[178735]: from='client.64102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:14.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:14 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:14.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:07:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:07:14] "GET /metrics HTTP/1.1" 200 37926 "" "Prometheus/2.43.0" 2023-10-01T20:07:14.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:14 smithi057 ceph-mon[205388]: from='client.64102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:14.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:14 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:14.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:14 smithi057 ceph-mon[209060]: from='client.64102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:14.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:14 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:15.451 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T20:07:15.452 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 8s ago 27m - - 2023-10-01T20:07:15.452 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 17s ago 27m - - 2023-10-01T20:07:15.452 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (2m) 8s ago 23m - - 0.25.0 c8568f914cd2 ee46304105ca 2023-10-01T20:07:15.452 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (2m) 17s ago 23m 84.9M - 9.4.7 2c41d148cca3 e676c0004015 2023-10-01T20:07:15.452 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (3m) 8s ago 23m - - 3.5 e1d6a67b021e 7ef2818f7c68 2023-10-01T20:07:15.452 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283,8765 running (3m) 17s ago 27m 447M - 18.0.0-6505-g37d71fd3 bea6d6056140 b5797dd6a1b2 2023-10-01T20:07:15.452 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (13m) 8s ago 28m - - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T20:07:15.452 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (41s) 8s ago 28m - 2048M 18.0.0-6505-g37d71fd3 bea6d6056140 99dcb8cd4b2a 2023-10-01T20:07:15.452 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (108s) 17s ago 27m 66.8M 2048M 18.0.0-6505-g37d71fd3 bea6d6056140 805778222e17 2023-10-01T20:07:15.452 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (26s) 8s ago 27m - 2048M 18.0.0-6505-g37d71fd3 bea6d6056140 8425f2f44a60 2023-10-01T20:07:15.452 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (3m) 8s ago 24m - - 1.5.0 0da6a335fe13 175d4990d5fd 2023-10-01T20:07:15.453 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (3m) 17s ago 24m 18.2M - 1.5.0 0da6a335fe13 b2605eeb2dd4 2023-10-01T20:07:15.453 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (26m) 8s ago 26m - 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T20:07:15.453 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (26m) 8s ago 26m - 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T20:07:15.453 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (26m) 8s ago 26m - 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T20:07:15.453 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (25m) 8s ago 25m - 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T20:07:15.453 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (25m) 17s ago 25m 198M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T20:07:15.453 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (25m) 17s ago 25m 189M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T20:07:15.453 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (25m) 17s ago 25m 151M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T20:07:15.453 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (24m) 17s ago 24m 186M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T20:07:15.453 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (3m) 17s ago 24m 60.3M - 2.43.0 a07b618ecd1d 79b0216ed806 2023-10-01T20:07:15.453 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (23m) 8s ago 23m - - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T20:07:15.453 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (23m) 17s ago 23m 97.5M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T20:07:15.630 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:15 smithi057 ceph-mon[205388]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:15.631 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:15 smithi057 ceph-mon[209060]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:15.847 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions | jq -e '"'"'.mon | length == 1'"'"'' 2023-10-01T20:07:15.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:15 smithi087 ceph-mon[178735]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:16.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:16 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:16.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:16 smithi087 ceph-mon[178735]: from='client.64108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:16.902 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:16 smithi057 ceph-mon[205388]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:16.902 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:16 smithi057 ceph-mon[205388]: from='client.64108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:16.903 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:16 smithi057 ceph-mon[209060]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:16.903 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:16 smithi057 ceph-mon[209060]: from='client.64108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:17.646 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:07:17.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:17 smithi057 ceph-mon[205388]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:17.873 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:17 smithi057 ceph-mon[209060]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:17.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:17 smithi087 ceph-mon[178735]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:18.069 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions | jq -e '"'"'.mon | keys'"'"' | grep $sha1' 2023-10-01T20:07:18.845 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:18 smithi057 ceph-mon[205388]: from='client.? 172.21.15.57:0/173679010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:18.845 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:18 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:18.846 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:18 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:18.846 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:18 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:18.846 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:18 smithi057 ceph-mon[205388]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:18.846 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:18 smithi057 ceph-mon[209060]: from='client.? 172.21.15.57:0/173679010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:18.846 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:18 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:18.846 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:18 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:18.846 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:18 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:18.846 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:18 smithi057 ceph-mon[209060]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:18.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:18 smithi087 ceph-mon[178735]: from='client.? 172.21.15.57:0/173679010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:18.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:18 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:18.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:18 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:18.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:18 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:18.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:18 smithi087 ceph-mon[178735]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:19.795 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)" 2023-10-01T20:07:20.167 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 5'"'"'' 2023-10-01T20:07:20.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:20 smithi057 ceph-mon[205388]: from='client.? 172.21.15.57:0/3912468123' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:20.889 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:20 smithi057 ceph-mon[205388]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:20.889 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:20 smithi057 ceph-mon[209060]: from='client.? 172.21.15.57:0/3912468123' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:20.889 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:20 smithi057 ceph-mon[209060]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:20 smithi087 ceph-mon[178735]: from='client.? 172.21.15.57:0/3912468123' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:21.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:20 smithi087 ceph-mon[178735]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:23.293 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:07:23.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:23 smithi087 ceph-mon[178735]: from='client.54153 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:23.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:23 smithi087 ceph-mon[178735]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:23.669 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:23 smithi057 ceph-mon[205388]: from='client.54153 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:23.669 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:23 smithi057 ceph-mon[205388]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:23.669 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:23 smithi057 ceph-mon[209060]: from='client.54153 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:23.669 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:23 smithi057 ceph-mon[209060]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:23.691 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch upgrade status' 2023-10-01T20:07:24.841 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:07:24 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:07:24] "GET /metrics HTTP/1.1" 200 37928 "" "Prometheus/2.43.0" 2023-10-01T20:07:25.301 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:07:25.302 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": null, 2023-10-01T20:07:25.302 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": false, 2023-10-01T20:07:25.302 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "", 2023-10-01T20:07:25.302 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2023-10-01T20:07:25.302 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": null, 2023-10-01T20:07:25.302 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "", 2023-10-01T20:07:25.302 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2023-10-01T20:07:25.302 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:07:25.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:25 smithi087 ceph-mon[178735]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:25.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:25 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:07:25.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:25 smithi057 ceph-mon[205388]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:25.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:25 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:07:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:25 smithi057 ceph-mon[209060]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:25.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:25 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:07:25.732 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph health detail' 2023-10-01T20:07:26.375 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:26 smithi057 ceph-mon[209060]: from='client.54159 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:26.375 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:26 smithi057 ceph-mon[209060]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:26 smithi087 ceph-mon[178735]: from='client.54159 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:26.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:26 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:26.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:26 smithi057 ceph-mon[205388]: from='client.54159 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:26.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:26 smithi057 ceph-mon[205388]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:27.645 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:27 smithi057 ceph-mon[205388]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:27.645 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:27 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:27.646 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:27 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:27.646 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:27 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:27.646 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:27 smithi057 ceph-mon[209060]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:27.646 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:27 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:27.646 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:27 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:27.646 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:27 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:27.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:27 smithi087 ceph-mon[178735]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:27.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:27 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:27.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:27 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:27.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:27 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:28.120 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2023-10-01T20:07:28.503 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types osd --limit 2' 2023-10-01T20:07:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:28 smithi057 ceph-mon[205388]: from='client.? 172.21.15.57:0/3835329030' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T20:07:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:28 smithi057 ceph-mon[209060]: from='client.? 172.21.15.57:0/3835329030' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T20:07:28.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:28 smithi087 ceph-mon[178735]: from='client.? 172.21.15.57:0/3835329030' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2023-10-01T20:07:29.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:29 smithi057 ceph-mon[205388]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:29.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:29 smithi057 ceph-mon[209060]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:29.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:29 smithi087 ceph-mon[178735]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:31.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:30 smithi087 ceph-mon[178735]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:31.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:30 smithi087 ceph-mon[178735]: from='client.64144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "daemon_types": "osd", "limit": 2, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:31.169 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:30 smithi057 ceph-mon[205388]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:31.170 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:30 smithi057 ceph-mon[205388]: from='client.64144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "daemon_types": "osd", "limit": 2, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:31.170 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:30 smithi057 ceph-mon[209060]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:31.170 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:30 smithi057 ceph-mon[209060]: from='client.64144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "daemon_types": "osd", "limit": 2, "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:31.851 INFO:teuthology.orchestra.run.smithi057.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:07:32.245 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2023-10-01T20:07:32.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[209060]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:07:32.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:32.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:32.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:32.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:32.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:32.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[209060]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:07:32.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[209060]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[205388]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:07:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[205388]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:07:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:32 smithi057 ceph-mon[205388]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:33.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:32 smithi087 ceph-mon[178735]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:07:33.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:32 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:33.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:32 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:33.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:32 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:33.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:32 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:33.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:32 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:33.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:32 smithi087 ceph-mon[178735]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9 2023-10-01T20:07:33.134 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:32 smithi087 ceph-mon[178735]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:33.886 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:07:34.629 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T20:07:34.630 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 7s ago 28m - - 2023-10-01T20:07:34.630 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 16s ago 27m - - 2023-10-01T20:07:34.630 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (3m) 7s ago 24m - - 0.25.0 c8568f914cd2 ee46304105ca 2023-10-01T20:07:34.630 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (3m) 16s ago 24m 87.3M - 9.4.7 2c41d148cca3 e676c0004015 2023-10-01T20:07:34.630 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (4m) 7s ago 23m - - 3.5 e1d6a67b021e 7ef2818f7c68 2023-10-01T20:07:34.630 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283,8765 running (3m) 16s ago 27m 447M - 18.0.0-6505-g37d71fd3 bea6d6056140 b5797dd6a1b2 2023-10-01T20:07:34.630 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (13m) 7s ago 28m - - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T20:07:34.630 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (60s) 7s ago 28m - 2048M 18.0.0-6505-g37d71fd3 bea6d6056140 99dcb8cd4b2a 2023-10-01T20:07:34.630 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (2m) 16s ago 27m 67.3M 2048M 18.0.0-6505-g37d71fd3 bea6d6056140 805778222e17 2023-10-01T20:07:34.630 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (46s) 7s ago 27m - 2048M 18.0.0-6505-g37d71fd3 bea6d6056140 8425f2f44a60 2023-10-01T20:07:34.631 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (3m) 7s ago 24m - - 1.5.0 0da6a335fe13 175d4990d5fd 2023-10-01T20:07:34.631 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (3m) 16s ago 24m 18.2M - 1.5.0 0da6a335fe13 b2605eeb2dd4 2023-10-01T20:07:34.631 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (27m) 7s ago 27m - 4096M 17.2.0 e1d6a67b021e e0751e66db50 2023-10-01T20:07:34.631 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (26m) 7s ago 26m - 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T20:07:34.631 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (26m) 7s ago 26m - 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T20:07:34.631 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (26m) 7s ago 26m - 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T20:07:34.631 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (25m) 16s ago 25m 200M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T20:07:34.631 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (25m) 16s ago 25m 190M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T20:07:34.631 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (25m) 16s ago 25m 152M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T20:07:34.631 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (25m) 16s ago 25m 187M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T20:07:34.631 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (3m) 16s ago 24m 60.4M - 2.43.0 a07b618ecd1d 79b0216ed806 2023-10-01T20:07:34.631 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (23m) 7s ago 23m - - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T20:07:34.632 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (23m) 16s ago 23m 97.6M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T20:07:34.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:07:34] "GET /metrics HTTP/1.1" 200 37928 "" "Prometheus/2.43.0" 2023-10-01T20:07:35.068 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:07:35.069 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2023-10-01T20:07:35.069 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 3 2023-10-01T20:07:35.069 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:07:35.069 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2023-10-01T20:07:35.069 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 2 2023-10-01T20:07:35.069 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:07:35.069 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2023-10-01T20:07:35.069 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2023-10-01T20:07:35.069 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:07:35.069 INFO:teuthology.orchestra.run.smithi057.stdout: "rgw": { 2023-10-01T20:07:35.069 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2023-10-01T20:07:35.069 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-01T20:07:35.070 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2023-10-01T20:07:35.070 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 10, 2023-10-01T20:07:35.070 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.0.0-6505-g37d71fd3 (37d71fd39f09a164b3518d8aef9e20133197e5f9) reef (dev)": 5 2023-10-01T20:07:35.070 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-01T20:07:35.070 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:07:35.338 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: from='client.64150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:35.338 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:35.338 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: Upgrade: Target is version 18.0.0-6505-g37d71fd3 (reef) 2023-10-01T20:07:35.338 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24, digests ['quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24'] 2023-10-01T20:07:35.338 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:35.338 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:35.339 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all mgr 2023-10-01T20:07:35.339 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:35.339 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:35.339 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all mon 2023-10-01T20:07:35.339 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:35.339 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:35.339 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: Upgrade: Setting container_image for all crash 2023-10-01T20:07:35.339 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:35.339 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2023-10-01T20:07:35.339 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: from='mon.2 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2023-10-01T20:07:35.339 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: Upgrade: osd.0 is safe to restart 2023-10-01T20:07:35.339 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: from='client.64156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:35.340 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[209060]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:35.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: from='client.64150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:35.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:35.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: Upgrade: Target is version 18.0.0-6505-g37d71fd3 (reef) 2023-10-01T20:07:35.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24, digests ['quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24'] 2023-10-01T20:07:35.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:35.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:35.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all mgr 2023-10-01T20:07:35.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:35.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:35.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all mon 2023-10-01T20:07:35.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:35.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:35.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: Upgrade: Setting container_image for all crash 2023-10-01T20:07:35.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:35.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2023-10-01T20:07:35.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: from='mon.2 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2023-10-01T20:07:35.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: Upgrade: osd.0 is safe to restart 2023-10-01T20:07:35.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: from='client.64156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:35.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:34 smithi057 ceph-mon[205388]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: from='client.64150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: Upgrade: Target is version 18.0.0-6505-g37d71fd3 (reef) 2023-10-01T20:07:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24, digests ['quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24'] 2023-10-01T20:07:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:35.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:35.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all mgr 2023-10-01T20:07:35.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:35.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:35.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all mon 2023-10-01T20:07:35.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:35.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:35.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: Upgrade: Setting container_image for all crash 2023-10-01T20:07:35.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:35.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2023-10-01T20:07:35.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: from='mon.2 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2023-10-01T20:07:35.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: Upgrade: osd.0 is safe to restart 2023-10-01T20:07:35.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: from='client.64156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:35.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:34 smithi087 ceph-mon[178735]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:35.448 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-01T20:07:35.448 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:77acc4c088be6af3a1febbb71c133f46cf301e630beda5ebf693dedd0cbb6c24", 2023-10-01T20:07:35.448 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2023-10-01T20:07:35.448 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) osd. Upgrade limited to 2 daemons (2 remaining).", 2023-10-01T20:07:35.448 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2023-10-01T20:07:35.448 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "0/8 daemons upgraded", 2023-10-01T20:07:35.448 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading osd daemons", 2023-10-01T20:07:35.448 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2023-10-01T20:07:35.448 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-01T20:07:36.116 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:36 smithi087 ceph-mon[178735]: from='client.64162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:36.117 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:36 smithi087 ceph-mon[178735]: Upgrade: Updating osd.0 2023-10-01T20:07:36.117 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:36 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:36.117 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:36 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-01T20:07:36.117 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:36 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:36.117 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:36 smithi087 ceph-mon[178735]: Deploying daemon osd.0 on smithi057 2023-10-01T20:07:36.117 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:36 smithi087 ceph-mon[178735]: from='client.? 172.21.15.57:0/814476236' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:36.117 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:36 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:36.117 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:36 smithi087 ceph-mon[178735]: from='client.64174 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:36.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[205388]: from='client.64162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:36.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[205388]: Upgrade: Updating osd.0 2023-10-01T20:07:36.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:36.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-01T20:07:36.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:36.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[205388]: Deploying daemon osd.0 on smithi057 2023-10-01T20:07:36.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[205388]: from='client.? 172.21.15.57:0/814476236' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:36.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[205388]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:36.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[205388]: from='client.64174 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[209060]: from='client.64162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[209060]: Upgrade: Updating osd.0 2023-10-01T20:07:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:36.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-01T20:07:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[209060]: Deploying daemon osd.0 on smithi057 2023-10-01T20:07:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[209060]: from='client.? 172.21.15.57:0/814476236' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[209060]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:36.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:36 smithi057 ceph-mon[209060]: from='client.64174 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:07:37.068 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:37 smithi057 ceph-mon[205388]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:37.068 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:37 smithi057 ceph-mon[209060]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:37.349 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:37 smithi087 ceph-mon[178735]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-10-01T20:07:37.366 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:37 smithi057 systemd[1]: Stopping Ceph osd.0 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:07:37.671 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:37 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[118280]: 2023-10-01T20:07:37.364+0000 7f8f85892700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:07:37.672 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:37 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[118280]: 2023-10-01T20:07:37.364+0000 7f8f85892700 -1 osd.0 108 *** Got signal Terminated *** 2023-10-01T20:07:37.672 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:37 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[118280]: 2023-10-01T20:07:37.364+0000 7f8f85892700 -1 osd.0 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-01T20:07:38.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:38 smithi087 ceph-mon[178735]: osd.0 marked itself down and dead 2023-10-01T20:07:38.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:38 smithi057 ceph-mon[205388]: osd.0 marked itself down and dead 2023-10-01T20:07:38.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:38 smithi057 ceph-mon[209060]: osd.0 marked itself down and dead 2023-10-01T20:07:38.922 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:38 smithi057 bash[214406]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0 2023-10-01T20:07:39.275 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[205388]: Health check failed: 1 osds down (OSD_DOWN) 2023-10-01T20:07:39.276 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[205388]: failed to encode map e109 with expected crc 2023-10-01T20:07:39.276 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[205388]: failed to encode map e109 with expected crc 2023-10-01T20:07:39.276 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[205388]: failed to encode map e109 with expected crc 2023-10-01T20:07:39.276 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[205388]: osdmap e109: 8 total, 7 up, 8 in 2023-10-01T20:07:39.276 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[205388]: failed to encode map e109 with expected crc 2023-10-01T20:07:39.276 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:39.276 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:39.276 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:39.276 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[205388]: pgmap v41: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-10-01T20:07:39.276 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[209060]: Health check failed: 1 osds down (OSD_DOWN) 2023-10-01T20:07:39.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[209060]: failed to encode map e109 with expected crc 2023-10-01T20:07:39.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[209060]: failed to encode map e109 with expected crc 2023-10-01T20:07:39.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[209060]: failed to encode map e109 with expected crc 2023-10-01T20:07:39.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[209060]: osdmap e109: 8 total, 7 up, 8 in 2023-10-01T20:07:39.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[209060]: failed to encode map e109 with expected crc 2023-10-01T20:07:39.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:39.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:39.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:39.277 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:39 smithi057 ceph-mon[209060]: pgmap v41: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-10-01T20:07:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:39 smithi087 ceph-mon[178735]: Health check failed: 1 osds down (OSD_DOWN) 2023-10-01T20:07:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:39 smithi087 ceph-mon[178735]: failed to encode map e109 with expected crc 2023-10-01T20:07:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:39 smithi087 ceph-mon[178735]: failed to encode map e109 with expected crc 2023-10-01T20:07:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:39 smithi087 ceph-mon[178735]: failed to encode map e109 with expected crc 2023-10-01T20:07:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:39 smithi087 ceph-mon[178735]: osdmap e109: 8 total, 7 up, 8 in 2023-10-01T20:07:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:39 smithi087 ceph-mon[178735]: failed to encode map e109 with expected crc 2023-10-01T20:07:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:39 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:39.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:39 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:39 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:39.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:39 smithi087 ceph-mon[178735]: pgmap v41: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-10-01T20:07:40.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:40 smithi087 ceph-mon[178735]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:40 smithi087 ceph-mon[178735]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:40 smithi087 ceph-mon[178735]: osdmap e110: 8 total, 7 up, 8 in 2023-10-01T20:07:40.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:40 smithi087 ceph-mon[178735]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:40 smithi087 ceph-mon[178735]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:40 smithi087 ceph-mon[178735]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:40 smithi087 ceph-mon[178735]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:40 smithi087 ceph-mon[178735]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:40 smithi087 ceph-mon[178735]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:40 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:40.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:40 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:07:40.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[205388]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[205388]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[205388]: osdmap e110: 8 total, 7 up, 8 in 2023-10-01T20:07:40.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[205388]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[205388]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[205388]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[205388]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[205388]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[205388]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:40.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:07:40.419 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:40 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.0.service: Succeeded. 2023-10-01T20:07:40.419 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:40 smithi057 systemd[1]: Stopped Ceph osd.0 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:07:40.419 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:40 smithi057 systemd[1]: Starting Ceph osd.0 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:07:40.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[209060]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[209060]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[209060]: osdmap e110: 8 total, 7 up, 8 in 2023-10-01T20:07:40.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[209060]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[209060]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[209060]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[209060]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[209060]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[209060]: failed to encode map e110 with expected crc 2023-10-01T20:07:40.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:40.420 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:40 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:07:41.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:41 smithi087 ceph-mon[178735]: pgmap v43: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 68/627 objects degraded (10.845%) 2023-10-01T20:07:41.422 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:41 smithi057 ceph-mon[209060]: pgmap v43: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 68/627 objects degraded (10.845%) 2023-10-01T20:07:41.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:41 smithi057 ceph-mon[205388]: pgmap v43: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 68/627 objects degraded (10.845%) 2023-10-01T20:07:41.422 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:40 smithi057 bash[214672]: --> Failed to activate via raw: did not find any matching OSD to activate 2023-10-01T20:07:41.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:40 smithi057 bash[214672]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-01T20:07:41.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:40 smithi057 bash[214672]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2023-10-01T20:07:41.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:40 smithi057 bash[214672]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2023-10-01T20:07:41.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:40 smithi057 bash[214672]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2023-10-01T20:07:41.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:40 smithi057 bash[214672]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2023-10-01T20:07:41.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:40 smithi057 bash[214672]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-01T20:07:41.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:40 smithi057 bash[214672]: --> ceph-volume lvm activate successful for osd ID: 0 2023-10-01T20:07:41.921 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:41 smithi057 ceph-osd[214976]: -- 172.21.15.57:0/3760630725 <== mon.1 v2:172.21.15.57:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d6c07e84e0 con 0x55d6c07b9000 2023-10-01T20:07:41.921 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:41 smithi057 bash[214938]: ac2be25bb201978711decde463cc44c9b8ef97974e44e874b915f4256f71962d 2023-10-01T20:07:41.922 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:41 smithi057 systemd[1]: Started Ceph osd.0 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:07:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: Health check failed: Degraded data redundancy: 68/627 objects degraded (10.845%), 22 pgs degraded (PG_DEGRADED) 2023-10-01T20:07:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:42 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:42.421 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[214953]: 2023-10-01T20:07:42.150+0000 7f313b640640 -1 Falling back to public interface 2023-10-01T20:07:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: Health check failed: Degraded data redundancy: 68/627 objects degraded (10.845%), 22 pgs degraded (PG_DEGRADED) 2023-10-01T20:07:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:42.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: Health check failed: Degraded data redundancy: 68/627 objects degraded (10.845%), 22 pgs degraded (PG_DEGRADED) 2023-10-01T20:07:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:42.424 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:42.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:42 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: pgmap v44: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 68/627 objects degraded (10.845%) 2023-10-01T20:07:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: pgmap v44: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 68/627 objects degraded (10.845%) 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.179 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:43 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: pgmap v44: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 68/627 objects degraded (10.845%) 2023-10-01T20:07:43.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:43.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:43.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:43.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:43.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:43.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:43.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:43 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:44 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.428 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.429 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:44.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:44.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:44.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:44.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:44.776 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:07:44] "GET /metrics HTTP/1.1" 200 37988 "" "Prometheus/2.43.0" 2023-10-01T20:07:45.074 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[214953]: 2023-10-01T20:07:44.876+0000 7f313b640640 -1 osd.0 0 read_superblock omap replica is missing. 2023-10-01T20:07:45.075 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:44 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[214953]: 2023-10-01T20:07:44.911+0000 7f313b640640 -1 osd.0 108 log_to_monitors true 2023-10-01T20:07:45.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: pgmap v45: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 68/627 objects degraded (10.845%) 2023-10-01T20:07:45.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.351 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.351 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.351 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.351 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.351 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.351 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.351 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.351 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.351 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.351 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: pgmap v45: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 68/627 objects degraded (10.845%) 2023-10-01T20:07:45.352 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.353 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.353 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.353 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.353 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.353 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.353 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.353 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.353 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.353 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.354 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.354 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.354 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.354 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.354 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.354 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.354 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='osd.0 [v2:172.21.15.57:6802/2761849867,v1:172.21.15.57:6803/2761849867]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T20:07:45.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T20:07:45.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.356 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='osd.0 [v2:172.21.15.57:6802/2761849867,v1:172.21.15.57:6803/2761849867]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T20:07:45.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T20:07:45.357 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.358 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: pgmap v45: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 68/627 objects degraded (10.845%) 2023-10-01T20:07:45.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:45.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='osd.0 [v2:172.21.15.57:6802/2761849867,v1:172.21.15.57:6803/2761849867]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T20:07:45.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T20:07:45.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:45.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:45.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:45.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:45.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:45 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.171 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:45 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[214953]: 2023-10-01T20:07:45.864+0000 7f312baaa700 -1 osd.0 108 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-01T20:07:46.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:46.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.177 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-01T20:07:46.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: failed to encode map e111 with expected crc 2023-10-01T20:07:46.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: osdmap e111: 8 total, 7 up, 8 in 2023-10-01T20:07:46.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='osd.0 [v2:172.21.15.57:6802/2761849867,v1:172.21.15.57:6803/2761849867]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T20:07:46.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T20:07:46.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.180 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-01T20:07:46.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: failed to encode map e111 with expected crc 2023-10-01T20:07:46.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: osdmap e111: 8 total, 7 up, 8 in 2023-10-01T20:07:46.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='osd.0 [v2:172.21.15.57:6802/2761849867,v1:172.21.15.57:6803/2761849867]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T20:07:46.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T20:07:46.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.182 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.182 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.182 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.182 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.182 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.182 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:46 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-01T20:07:46.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: failed to encode map e111 with expected crc 2023-10-01T20:07:46.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: osdmap e111: 8 total, 7 up, 8 in 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='osd.0 [v2:172.21.15.57:6802/2761849867,v1:172.21.15.57:6803/2761849867]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:46.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:46.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:46 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.368 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.368 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: failed to encode map e111 with expected crc 2023-10-01T20:07:47.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: pgmap v47: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 68/627 objects degraded (10.845%) 2023-10-01T20:07:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.372 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.372 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.372 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: failed to encode map e111 with expected crc 2023-10-01T20:07:47.373 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.373 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.373 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.373 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.373 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.373 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.374 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.374 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.374 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.374 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: pgmap v47: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 68/627 objects degraded (10.845%) 2023-10-01T20:07:47.374 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.374 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.375 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.375 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.375 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.375 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.375 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.375 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.376 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.376 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.376 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.376 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.376 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.377 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.377 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.377 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: Health check cleared: OSD_DOWN (was: 1 osds down) 2023-10-01T20:07:47.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: failed to encode map e112 with expected crc 2023-10-01T20:07:47.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:07:47.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: osd.0 [v2:172.21.15.57:6802/2761849867,v1:172.21.15.57:6803/2761849867] boot 2023-10-01T20:07:47.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: osdmap e112: 8 total, 8 up, 8 in 2023-10-01T20:07:47.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: failed to encode map e112 with expected crc 2023-10-01T20:07:47.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: failed to encode map e112 with expected crc 2023-10-01T20:07:47.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.381 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.381 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.381 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.381 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.381 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.381 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.382 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.382 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.382 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.382 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.382 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.382 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.382 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.383 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.383 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: Health check cleared: OSD_DOWN (was: 1 osds down) 2023-10-01T20:07:47.383 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: failed to encode map e112 with expected crc 2023-10-01T20:07:47.383 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:07:47.383 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: osd.0 [v2:172.21.15.57:6802/2761849867,v1:172.21.15.57:6803/2761849867] boot 2023-10-01T20:07:47.383 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: osdmap e112: 8 total, 8 up, 8 in 2023-10-01T20:07:47.384 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: failed to encode map e112 with expected crc 2023-10-01T20:07:47.384 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: failed to encode map e112 with expected crc 2023-10-01T20:07:47.384 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.384 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.384 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.385 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.385 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.385 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.385 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:47 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: failed to encode map e111 with expected crc 2023-10-01T20:07:47.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.386 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.387 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: pgmap v47: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 68/627 objects degraded (10.845%) 2023-10-01T20:07:47.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.388 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.389 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:47.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.390 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: Health check cleared: OSD_DOWN (was: 1 osds down) 2023-10-01T20:07:47.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: failed to encode map e112 with expected crc 2023-10-01T20:07:47.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:07:47.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: osd.0 [v2:172.21.15.57:6802/2761849867,v1:172.21.15.57:6803/2761849867] boot 2023-10-01T20:07:47.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: osdmap e112: 8 total, 8 up, 8 in 2023-10-01T20:07:47.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: failed to encode map e112 with expected crc 2023-10-01T20:07:47.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: failed to encode map e112 with expected crc 2023-10-01T20:07:47.391 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:47.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:47.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:47.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:47.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:47.392 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:47 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.295 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:48.295 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:48.295 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:48.295 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.296 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.296 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:48.296 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:48.296 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:48.296 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:48.296 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2023-10-01T20:07:48.296 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: from='mon.2 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2023-10-01T20:07:48.296 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: Upgrade: osd.0 is safe to restart 2023-10-01T20:07:48.296 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.296 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.296 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.297 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:48.297 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: osdmap e113: 8 total, 8 up, 8 in 2023-10-01T20:07:48.297 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:48.297 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:48.297 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:48.297 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:48.297 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:48.297 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:48.297 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.297 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.297 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:48.298 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:48.298 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:48.298 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:48.298 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2023-10-01T20:07:48.298 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: from='mon.2 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2023-10-01T20:07:48.298 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: Upgrade: osd.0 is safe to restart 2023-10-01T20:07:48.298 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.298 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.298 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.298 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:48.298 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: osdmap e113: 8 total, 8 up, 8 in 2023-10-01T20:07:48.298 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:48.299 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:48.299 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:48 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:48.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2023-10-01T20:07:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: from='mon.2 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2023-10-01T20:07:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: Upgrade: osd.0 is safe to restart 2023-10-01T20:07:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: osdmap e113: 8 total, 8 up, 8 in 2023-10-01T20:07:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:48.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:48.385 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:48 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: pgmap v50: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:07:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-01T20:07:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:49.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:49.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:49.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:49.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: pgmap v50: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:07:49.174 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-01T20:07:49.175 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:49 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.383 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: failed to encode map e113 with expected crc 2023-10-01T20:07:49.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:49.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:49.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:49.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: pgmap v50: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:07:49.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-01T20:07:49.384 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:49 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:50.555 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:50 smithi057 ceph-mon[209060]: Upgrade: Updating osd.0 2023-10-01T20:07:50.555 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:50 smithi057 ceph-mon[209060]: Deploying daemon osd.0 on smithi057 2023-10-01T20:07:50.555 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:50 smithi057 ceph-mon[205388]: Upgrade: Updating osd.0 2023-10-01T20:07:50.556 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:50 smithi057 ceph-mon[205388]: Deploying daemon osd.0 on smithi057 2023-10-01T20:07:50.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:50 smithi087 ceph-mon[178735]: Upgrade: Updating osd.0 2023-10-01T20:07:50.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:50 smithi087 ceph-mon[178735]: Deploying daemon osd.0 on smithi057 2023-10-01T20:07:51.472 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:51 smithi057 ceph-mon[205388]: pgmap v51: 161 pgs: 8 active+undersized+degraded, 13 active+undersized, 140 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 29/627 objects degraded (4.625%) 2023-10-01T20:07:51.472 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:51 smithi057 ceph-mon[209060]: pgmap v51: 161 pgs: 8 active+undersized+degraded, 13 active+undersized, 140 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 29/627 objects degraded (4.625%) 2023-10-01T20:07:51.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:51 smithi087 ceph-mon[178735]: pgmap v51: 161 pgs: 8 active+undersized+degraded, 13 active+undersized, 140 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 29/627 objects degraded (4.625%) 2023-10-01T20:07:52.081 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:51 smithi057 systemd[1]: Stopping Ceph osd.0 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:07:52.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:52 smithi057 ceph-mon[205388]: Health check update: Degraded data redundancy: 29/627 objects degraded (4.625%), 8 pgs degraded (PG_DEGRADED) 2023-10-01T20:07:52.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:52 smithi057 ceph-mon[205388]: osd.0 marked itself down and dead 2023-10-01T20:07:52.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:52 smithi057 ceph-mon[209060]: Health check update: Degraded data redundancy: 29/627 objects degraded (4.625%), 8 pgs degraded (PG_DEGRADED) 2023-10-01T20:07:52.423 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:52 smithi057 ceph-mon[209060]: osd.0 marked itself down and dead 2023-10-01T20:07:52.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:52 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[214953]: 2023-10-01T20:07:52.079+0000 7f3131cb9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:07:52.424 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:52 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[214953]: 2023-10-01T20:07:52.079+0000 7f3131cb9700 -1 osd.0 113 *** Got signal Terminated *** 2023-10-01T20:07:52.424 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:52 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[214953]: 2023-10-01T20:07:52.079+0000 7f3131cb9700 -1 osd.0 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-01T20:07:52.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:52 smithi087 ceph-mon[178735]: Health check update: Degraded data redundancy: 29/627 objects degraded (4.625%), 8 pgs degraded (PG_DEGRADED) 2023-10-01T20:07:52.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:52 smithi087 ceph-mon[178735]: osd.0 marked itself down and dead 2023-10-01T20:07:53.171 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:52 smithi057 bash[218401]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0 2023-10-01T20:07:53.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:53 smithi087 ceph-mon[178735]: Health check failed: 1 osds down (OSD_DOWN) 2023-10-01T20:07:53.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:53 smithi087 ceph-mon[178735]: failed to encode map e114 with expected crc 2023-10-01T20:07:53.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:53 smithi087 ceph-mon[178735]: failed to encode map e114 with expected crc 2023-10-01T20:07:53.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:53 smithi087 ceph-mon[178735]: osdmap e114: 8 total, 7 up, 8 in 2023-10-01T20:07:53.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:53 smithi087 ceph-mon[178735]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:53.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:53 smithi087 ceph-mon[178735]: failed to encode map e114 with expected crc 2023-10-01T20:07:53.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:53 smithi087 ceph-mon[178735]: failed to encode map e114 with expected crc 2023-10-01T20:07:53.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[205388]: Health check failed: 1 osds down (OSD_DOWN) 2023-10-01T20:07:53.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[205388]: failed to encode map e114 with expected crc 2023-10-01T20:07:53.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[205388]: failed to encode map e114 with expected crc 2023-10-01T20:07:53.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[205388]: osdmap e114: 8 total, 7 up, 8 in 2023-10-01T20:07:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[205388]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[205388]: failed to encode map e114 with expected crc 2023-10-01T20:07:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[205388]: failed to encode map e114 with expected crc 2023-10-01T20:07:53.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[209060]: Health check failed: 1 osds down (OSD_DOWN) 2023-10-01T20:07:53.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[209060]: failed to encode map e114 with expected crc 2023-10-01T20:07:53.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[209060]: failed to encode map e114 with expected crc 2023-10-01T20:07:53.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[209060]: osdmap e114: 8 total, 7 up, 8 in 2023-10-01T20:07:53.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[209060]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-10-01T20:07:53.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[209060]: failed to encode map e114 with expected crc 2023-10-01T20:07:53.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:53 smithi057 ceph-mon[209060]: failed to encode map e114 with expected crc 2023-10-01T20:07:54.422 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:54 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.0.service: Succeeded. 2023-10-01T20:07:54.422 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:54 smithi057 systemd[1]: Stopped Ceph osd.0 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:07:54.422 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:54 smithi057 systemd[1]: Starting Ceph osd.0 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:07:54.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[205388]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/627 objects degraded (4.625%), 8 pgs degraded) 2023-10-01T20:07:54.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[205388]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[205388]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[205388]: osdmap e115: 8 total, 7 up, 8 in 2023-10-01T20:07:54.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[205388]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[205388]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[205388]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[205388]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[205388]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[205388]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[209060]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/627 objects degraded (4.625%), 8 pgs degraded) 2023-10-01T20:07:54.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[209060]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[209060]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[209060]: osdmap e115: 8 total, 7 up, 8 in 2023-10-01T20:07:54.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[209060]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[209060]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.425 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[209060]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[209060]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[209060]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.426 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-mon[209060]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:54 smithi087 ceph-mon[178735]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/627 objects degraded (4.625%), 8 pgs degraded) 2023-10-01T20:07:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:54 smithi087 ceph-mon[178735]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:54 smithi087 ceph-mon[178735]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:54 smithi087 ceph-mon[178735]: osdmap e115: 8 total, 7 up, 8 in 2023-10-01T20:07:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:54 smithi087 ceph-mon[178735]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:54 smithi087 ceph-mon[178735]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:54 smithi087 ceph-mon[178735]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:54 smithi087 ceph-mon[178735]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:54 smithi087 ceph-mon[178735]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:54 smithi087 ceph-mon[178735]: failed to encode map e115 with expected crc 2023-10-01T20:07:54.921 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:07:54 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:07:54] "GET /metrics HTTP/1.1" 200 37995 "" "Prometheus/2.43.0" 2023-10-01T20:07:55.301 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:55 smithi057 bash[218667]: --> Failed to activate via raw: did not find any matching OSD to activate 2023-10-01T20:07:55.301 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:55 smithi057 bash[218667]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-01T20:07:55.301 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:55 smithi057 bash[218667]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2023-10-01T20:07:55.301 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:55 smithi057 bash[218667]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2023-10-01T20:07:55.301 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:55 smithi057 bash[218667]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2023-10-01T20:07:55.301 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:55 smithi057 bash[218667]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2023-10-01T20:07:55.301 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:55 smithi057 bash[218667]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-01T20:07:55.301 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:55 smithi057 bash[218667]: --> ceph-volume lvm activate successful for osd ID: 0 2023-10-01T20:07:55.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:55 smithi087 ceph-mon[178735]: pgmap v55: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 792 B/s rd, 0 op/s 2023-10-01T20:07:55.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:55 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:07:55.662 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:55 smithi057 ceph-mon[205388]: pgmap v55: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 792 B/s rd, 0 op/s 2023-10-01T20:07:55.662 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:55 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:07:55.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:55 smithi057 ceph-mon[209060]: pgmap v55: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 792 B/s rd, 0 op/s 2023-10-01T20:07:55.663 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:55 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:07:56.171 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:55 smithi057 ceph-osd[218966]: -- 172.21.15.57:0/1977841652 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56302e1b64e0 con 0x56302e186c00 2023-10-01T20:07:56.172 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:55 smithi057 bash[218929]: 630435c91fd9dbe36c0de9d2c22715c46a2491b477d932545a8b7752e4ecd7ce 2023-10-01T20:07:56.172 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:55 smithi057 systemd[1]: Started Ceph osd.0 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:07:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:56.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:56.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:56.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:56.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:56 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:56.671 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[218943]: 2023-10-01T20:07:56.237+0000 7fe83d77d640 -1 Falling back to public interface 2023-10-01T20:07:56.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:56.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:56.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:56.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:56.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:56.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:56.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:07:56.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.674 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:56.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:56.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:56.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:56.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:56.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:56.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:56.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:56.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:56.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:56.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:56.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:56.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:56 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: pgmap v56: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:07:57.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: Health check failed: Degraded data redundancy: 68/627 objects degraded (10.845%), 22 pgs degraded (PG_DEGRADED) 2023-10-01T20:07:57.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:57 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: pgmap v56: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:07:57.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: pgmap v56: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:07:57.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.675 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: Health check failed: Degraded data redundancy: 68/627 objects degraded (10.845%), 22 pgs degraded (PG_DEGRADED) 2023-10-01T20:07:57.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:57.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: Health check failed: Degraded data redundancy: 68/627 objects degraded (10.845%), 22 pgs degraded (PG_DEGRADED) 2023-10-01T20:07:57.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:57.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:57.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:57.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:57.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:57 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.333 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.333 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.333 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.333 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.334 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.334 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.334 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.334 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.334 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.334 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.334 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.633 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.634 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.635 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.637 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.638 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:58 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:58.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:58.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:58.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:58.682 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:58.683 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:58.683 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.289 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:58 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[218943]: 2023-10-01T20:07:58.993+0000 7fe83d77d640 -1 osd.0 113 log_to_monitors true 2023-10-01T20:07:59.654 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.655 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.655 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.655 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.655 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: pgmap v57: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:07:59.655 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.655 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.655 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.655 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.655 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.655 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:59.655 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.655 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:59.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:59.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:59.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:59.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:59.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='osd.0 [v2:172.21.15.57:6802/282914384,v1:172.21.15.57:6803/282914384]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T20:07:59.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T20:07:59.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:59.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:59.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:59.657 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.657 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.657 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.657 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.657 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: pgmap v57: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:07:59.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:59.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:59.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:59.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.658 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.659 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:59.659 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.659 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:59.659 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:59.659 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.659 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='osd.0 [v2:172.21.15.57:6802/282914384,v1:172.21.15.57:6803/282914384]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T20:07:59.659 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T20:07:59.659 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.659 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:59.659 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.659 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:59.659 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:59.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:59.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:59.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:59.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.661 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.661 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.661 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:59.661 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.661 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:59.661 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:59.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.662 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.662 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:07:59 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[218943]: 2023-10-01T20:07:59.652+0000 7fe82b3e2700 -1 osd.0 113 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-01T20:07:59.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: pgmap v57: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:07:59.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:59.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:59.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:59.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:07:59.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:59.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:59.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:59.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='osd.0 [v2:172.21.15.57:6802/282914384,v1:172.21.15.57:6803/282914384]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T20:07:59.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-01T20:07:59.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:59.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:59.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:59.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:07:59.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:07:59.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:07:59.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:07:59.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:07:59.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:07:59 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.405 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:00.405 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:00.405 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:00.405 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.405 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.405 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.405 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.405 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.405 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: failed to encode map e116 with expected crc 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: failed to encode map e116 with expected crc 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: osdmap e116: 8 total, 7 up, 8 in 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='osd.0 [v2:172.21.15.57:6802/282914384,v1:172.21.15.57:6803/282914384]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.406 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.407 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: failed to encode map e116 with expected crc 2023-10-01T20:08:00.407 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.407 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.407 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.407 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.407 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.407 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.407 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: failed to encode map e116 with expected crc 2023-10-01T20:08:00.407 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.407 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.407 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.407 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.408 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.408 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.408 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.408 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.408 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:00.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:00.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:00.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:00.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-01T20:08:00.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: failed to encode map e116 with expected crc 2023-10-01T20:08:00.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: failed to encode map e116 with expected crc 2023-10-01T20:08:00.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: osdmap e116: 8 total, 7 up, 8 in 2023-10-01T20:08:00.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='osd.0 [v2:172.21.15.57:6802/282914384,v1:172.21.15.57:6803/282914384]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T20:08:00.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T20:08:00.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: failed to encode map e116 with expected crc 2023-10-01T20:08:00.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: failed to encode map e116 with expected crc 2023-10-01T20:08:00.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.676 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:00 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-01T20:08:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: failed to encode map e116 with expected crc 2023-10-01T20:08:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: failed to encode map e116 with expected crc 2023-10-01T20:08:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: osdmap e116: 8 total, 7 up, 8 in 2023-10-01T20:08:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='osd.0 [v2:172.21.15.57:6802/282914384,v1:172.21.15.57:6803/282914384]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T20:08:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-01T20:08:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: failed to encode map e116 with expected crc 2023-10-01T20:08:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: failed to encode map e116 with expected crc 2023-10-01T20:08:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:00.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:00 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.404 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.404 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.404 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: failed to encode map e116 with expected crc 2023-10-01T20:08:01.404 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.404 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.404 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: pgmap v59: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:08:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: Health check cleared: OSD_DOWN (was: 1 osds down) 2023-10-01T20:08:01.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: failed to encode map e117 with expected crc 2023-10-01T20:08:01.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: osd.0 [v2:172.21.15.57:6802/282914384,v1:172.21.15.57:6803/282914384] boot 2023-10-01T20:08:01.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: osdmap e117: 8 total, 8 up, 8 in 2023-10-01T20:08:01.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: failed to encode map e117 with expected crc 2023-10-01T20:08:01.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:08:01.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: failed to encode map e116 with expected crc 2023-10-01T20:08:01.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.677 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: pgmap v59: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:08:01.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.678 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: Health check cleared: OSD_DOWN (was: 1 osds down) 2023-10-01T20:08:01.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: failed to encode map e117 with expected crc 2023-10-01T20:08:01.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: osd.0 [v2:172.21.15.57:6802/282914384,v1:172.21.15.57:6803/282914384] boot 2023-10-01T20:08:01.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: osdmap e117: 8 total, 8 up, 8 in 2023-10-01T20:08:01.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: failed to encode map e117 with expected crc 2023-10-01T20:08:01.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:08:01.679 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.680 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.681 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:01 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: failed to encode map e116 with expected crc 2023-10-01T20:08:01.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: pgmap v59: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:08:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.884 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: Health check cleared: OSD_DOWN (was: 1 osds down) 2023-10-01T20:08:01.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: failed to encode map e117 with expected crc 2023-10-01T20:08:01.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: osd.0 [v2:172.21.15.57:6802/282914384,v1:172.21.15.57:6803/282914384] boot 2023-10-01T20:08:01.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: osdmap e117: 8 total, 8 up, 8 in 2023-10-01T20:08:01.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: failed to encode map e117 with expected crc 2023-10-01T20:08:01.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-01T20:08:01.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:01.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:01.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:01.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:01.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:01.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:01 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.885 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:02.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:02.886 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.887 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:08:02.888 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:08:02.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:08:02.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:08:02.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:08:02.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: osdmap e118: 8 total, 8 up, 8 in 2023-10-01T20:08:02.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:08:02.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.889 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:08:02.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:08:02.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:08:02.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:08:02.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:08:02.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:08:02.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:08:02.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:08:02.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.890 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.891 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.891 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.891 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.891 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.891 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.891 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.891 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.891 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.891 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.891 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.891 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:08:02.891 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:08:02.892 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2023-10-01T20:08:02.892 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.892 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:02.892 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:02.892 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.892 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.892 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:02.892 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:02.892 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.893 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.893 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.893 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:02 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.923 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:08:02.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:08:02.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2023-10-01T20:08:02.924 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2023-10-01T20:08:02.925 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:08:02.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: Metadata not up to date on all hosts. Skipping non agent specs 2023-10-01T20:08:02.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:02.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:02.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:02.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:08:02.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2023-10-01T20:08:02.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2023-10-01T20:08:02.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2023-10-01T20:08:02.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:08:02.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:08:02.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2023-10-01T20:08:02.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:08:02.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:08:02.929 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:08:02.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:08:02.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:08:02.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:08:02.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:08:02.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: osdmap e118: 8 total, 8 up, 8 in 2023-10-01T20:08:02.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:08:02.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.930 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:08:02.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:08:02.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:08:02.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:08:02.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:08:02.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:08:02.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:08:02.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:08:02.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:08:02.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:08:02.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2023-10-01T20:08:02.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:02.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:02.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:02.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:02.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.933 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2023-10-01T20:08:02.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2023-10-01T20:08:02.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:08:02.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2023-10-01T20:08:02.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:08:02.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2023-10-01T20:08:02.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:08:02.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2023-10-01T20:08:02.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:08:02.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: osdmap e118: 8 total, 8 up, 8 in 2023-10-01T20:08:02.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2023-10-01T20:08:02.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: failed to encode map e118 with expected crc 2023-10-01T20:08:02.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:08:02.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2023-10-01T20:08:02.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:08:02.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2023-10-01T20:08:02.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:08:02.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2023-10-01T20:08:02.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:08:02.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2023-10-01T20:08:02.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2023-10-01T20:08:02.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:08:02.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2023-10-01T20:08:02.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2023-10-01T20:08:02.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:02.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:02.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-01T20:08:02.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-01T20:08:02.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-01T20:08:02.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:02.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:02 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:03 smithi057 ceph-mon[209060]: Upgrade: Finalizing container_image settings 2023-10-01T20:08:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:03 smithi057 ceph-mon[209060]: failed to encode map e118 with expected crc 2023-10-01T20:08:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:03 smithi057 ceph-mon[209060]: Upgrade: Complete! 2023-10-01T20:08:03.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:03 smithi057 ceph-mon[209060]: pgmap v62: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:08:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:03 smithi057 ceph-mon[205388]: Upgrade: Finalizing container_image settings 2023-10-01T20:08:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:03 smithi057 ceph-mon[205388]: failed to encode map e118 with expected crc 2023-10-01T20:08:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:03 smithi057 ceph-mon[205388]: Upgrade: Complete! 2023-10-01T20:08:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:03 smithi057 ceph-mon[205388]: pgmap v62: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:08:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:03 smithi087 ceph-mon[178735]: Upgrade: Finalizing container_image settings 2023-10-01T20:08:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:03 smithi087 ceph-mon[178735]: failed to encode map e118 with expected crc 2023-10-01T20:08:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:03 smithi087 ceph-mon[178735]: Upgrade: Complete! 2023-10-01T20:08:03.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:03 smithi087 ceph-mon[178735]: pgmap v62: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:08:05.172 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:08:04 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:08:04] "GET /metrics HTTP/1.1" 200 37995 "" "Prometheus/2.43.0" 2023-10-01T20:08:05.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:05 smithi057 ceph-mon[205388]: pgmap v63: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:08:05.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:05 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:05.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:05 smithi057 ceph-mon[209060]: pgmap v63: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:08:05.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:05 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:05 smithi087 ceph-mon[178735]: pgmap v63: 161 pgs: 22 active+undersized+degraded, 36 active+undersized, 103 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 68/627 objects degraded (10.845%) 2023-10-01T20:08:05.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:05 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:06.195 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch ps' 2023-10-01T20:08:06.726 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:06 smithi057 ceph-mon[205388]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:08:06.726 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:06 smithi057 ceph-mon[209060]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:08:06.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:06 smithi087 ceph-mon[178735]: from='client.25321 -' entity='client.iscsi.foo.smithi057.zfdcts' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-10-01T20:08:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:07 smithi057 ceph-mon[205388]: from='client.54213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:08:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:07 smithi057 ceph-mon[205388]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2023-10-01T20:08:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:07 smithi057 ceph-mon[205388]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 68/627 objects degraded (10.845%), 22 pgs degraded) 2023-10-01T20:08:07.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:07 smithi057 ceph-mon[205388]: Cluster is now healthy 2023-10-01T20:08:07.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:07 smithi057 ceph-mon[209060]: from='client.54213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:08:07.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:07 smithi057 ceph-mon[209060]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2023-10-01T20:08:07.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:07 smithi057 ceph-mon[209060]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 68/627 objects degraded (10.845%), 22 pgs degraded) 2023-10-01T20:08:07.673 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:07 smithi057 ceph-mon[209060]: Cluster is now healthy 2023-10-01T20:08:07.699 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-01T20:08:07.700 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi057 smithi057 running 5s ago 28m - - 2023-10-01T20:08:07.700 INFO:teuthology.orchestra.run.smithi057.stdout:agent.smithi087 smithi087 running 9s ago 28m - - 2023-10-01T20:08:07.700 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.a smithi057 *:9093,9094 running (3m) 5s ago 24m 19.6M - 0.25.0 c8568f914cd2 ee46304105ca 2023-10-01T20:08:07.700 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.a smithi087 *:3000 running (3m) 9s ago 24m 94.1M - 9.4.7 2c41d148cca3 e676c0004015 2023-10-01T20:08:07.700 INFO:teuthology.orchestra.run.smithi057.stdout:iscsi.foo.smithi057.zfdcts smithi057 running (4m) 5s ago 24m 58.3M - 3.5 e1d6a67b021e 7ef2818f7c68 2023-10-01T20:08:07.700 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi087 *:8443,9283,8765 running (4m) 9s ago 27m 448M - 18.0.0-6505-g37d71fd3 bea6d6056140 b5797dd6a1b2 2023-10-01T20:08:07.700 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi057 *:8443,9283,8765 running (14m) 5s ago 29m 544M - 18.0.0-6505-g37d71fd3 bea6d6056140 50d5f27e09f0 2023-10-01T20:08:07.700 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (93s) 5s ago 29m 74.3M 2048M 18.0.0-6505-g37d71fd3 bea6d6056140 99dcb8cd4b2a 2023-10-01T20:08:07.700 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi087 running (2m) 9s ago 28m 75.5M 2048M 18.0.0-6505-g37d71fd3 bea6d6056140 805778222e17 2023-10-01T20:08:07.701 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi057 running (79s) 5s ago 28m 59.9M 2048M 18.0.0-6505-g37d71fd3 bea6d6056140 8425f2f44a60 2023-10-01T20:08:07.701 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.a smithi057 *:9100 running (4m) 5s ago 25m 9537k - 1.5.0 0da6a335fe13 175d4990d5fd 2023-10-01T20:08:07.701 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.b smithi087 *:9100 running (4m) 9s ago 25m 18.2M - 1.5.0 0da6a335fe13 b2605eeb2dd4 2023-10-01T20:08:07.701 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (12s) 5s ago 27m 51.1M 4096M 18.0.0-6505-g37d71fd3 bea6d6056140 630435c91fd9 2023-10-01T20:08:07.701 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (27m) 5s ago 27m 200M 4096M 17.2.0 e1d6a67b021e f588555781cd 2023-10-01T20:08:07.701 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (27m) 5s ago 27m 144M 4096M 17.2.0 e1d6a67b021e f7c79ec36bf8 2023-10-01T20:08:07.701 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (26m) 5s ago 26m 224M 4096M 17.2.0 e1d6a67b021e 7ed06ee322b3 2023-10-01T20:08:07.701 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi087 running (26m) 9s ago 26m 210M 4096M 17.2.0 e1d6a67b021e 7c746715858a 2023-10-01T20:08:07.701 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi087 running (26m) 9s ago 26m 202M 4096M 17.2.0 e1d6a67b021e c92fde6d6570 2023-10-01T20:08:07.701 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi087 running (26m) 9s ago 26m 160M 4096M 17.2.0 e1d6a67b021e 91101a24c4ad 2023-10-01T20:08:07.701 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi087 running (25m) 9s ago 25m 199M 4096M 17.2.0 e1d6a67b021e 95cb76809789 2023-10-01T20:08:07.702 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.a smithi087 *:9095 running (4m) 9s ago 25m 61.3M - 2.43.0 a07b618ecd1d 79b0216ed806 2023-10-01T20:08:07.702 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi057.dbidoa smithi057 *:8000 running (24m) 5s ago 24m 101M - 17.2.0 e1d6a67b021e f5fce597228a 2023-10-01T20:08:07.702 INFO:teuthology.orchestra.run.smithi057.stdout:rgw.foo.smithi087.rxeurp smithi087 *:8000 running (24m) 9s ago 24m 97.7M - 17.2.0 e1d6a67b021e 9be3d955530c 2023-10-01T20:08:07.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:07 smithi087 ceph-mon[178735]: from='client.54213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:08:07.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:07 smithi087 ceph-mon[178735]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2023-10-01T20:08:07.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:07 smithi087 ceph-mon[178735]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 68/627 objects degraded (10.845%), 22 pgs degraded) 2023-10-01T20:08:07.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:07 smithi087 ceph-mon[178735]: Cluster is now healthy 2023-10-01T20:08:08.077 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph versions | jq -e '"'"'.osd | length == 2'"'"'' 2023-10-01T20:08:09.657 INFO:teuthology.orchestra.run.smithi057.stdout:true 2023-10-01T20:08:09.814 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:09 smithi057 ceph-mon[205388]: from='client.64198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:08:09.814 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:09 smithi057 ceph-mon[205388]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2023-10-01T20:08:09.815 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:09 smithi057 ceph-mon[205388]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:09.815 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:09 smithi057 ceph-mon[205388]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:08:09.815 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:09 smithi057 ceph-mon[209060]: from='client.64198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:08:09.815 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:09 smithi057 ceph-mon[209060]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2023-10-01T20:08:09.815 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:09 smithi057 ceph-mon[209060]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:09.815 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:09 smithi057 ceph-mon[209060]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:08:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:09 smithi087 ceph-mon[178735]: from='client.64198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:08:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:09 smithi087 ceph-mon[178735]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2023-10-01T20:08:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:09 smithi087 ceph-mon[178735]: from='mgr.25561 ' entity='mgr.y' 2023-10-01T20:08:09.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:09 smithi087 ceph-mon[178735]: from='mgr.25561 172.21.15.57:0/2880295843' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-10-01T20:08:10.019 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 7'"'"'' 2023-10-01T20:08:10.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:10 smithi057 ceph-mon[205388]: from='client.? 172.21.15.57:0/1241481605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:10.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:10 smithi057 ceph-mon[209060]: from='client.? 172.21.15.57:0/1241481605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:10.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:10 smithi087 ceph-mon[178735]: from='client.? 172.21.15.57:0/1241481605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2023-10-01T20:08:11.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:11 smithi087 ceph-mon[178735]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2023-10-01T20:08:11.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:11 smithi057 ceph-mon[205388]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2023-10-01T20:08:11.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:11 smithi057 ceph-mon[209060]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2023-10-01T20:08:12.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:12 smithi057 ceph-mon[205388]: from='client.54231 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:08:12.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:12 smithi057 ceph-mon[209060]: from='client.54231 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:08:12.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:12 smithi087 ceph-mon[178735]: from='client.54231 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:37d71fd39f09a164b3518d8aef9e20133197e5f9", "target": ["mon-mgr", ""]}]: dispatch 2023-10-01T20:08:12.938 INFO:teuthology.orchestra.run.smithi057.stdout:false 2023-10-01T20:08:13.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-01T20:08:13.247 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_37d71fd39f09a164b3518d8aef9e20133197e5f9/qa/tasks/cephadm.py", line 1313, in shell _shell(ctx, cluster_name, remote, File "/home/teuthworker/src/git.ceph.com_ceph-c_37d71fd39f09a164b3518d8aef9e20133197e5f9/qa/tasks/cephadm.py", line 39, in _shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi057 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\'' 2023-10-01T20:08:13.315 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=933d3ec517e64d5599515ac11339f85d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_37d71fd39f09a164b3518d8aef9e20133197e5f9/qa/tasks/cephadm.py", line 1313, in shell _shell(ctx, cluster_name, remote, File "/home/teuthworker/src/git.ceph.com_ceph-c_37d71fd39f09a164b3518d8aef9e20133197e5f9/qa/tasks/cephadm.py", line 39, in _shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi057 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\'' 2023-10-01T20:08:13.318 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-01T20:08:13.327 INFO:tasks.cephadm:Teardown begin 2023-10-01T20:08:13.327 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-01T20:08:13.381 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-01T20:08:13.419 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-01T20:08:13.420 DEBUG:teuthology.orchestra.run.smithi057:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-01T20:08:13.444 DEBUG:teuthology.orchestra.run.smithi087:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-01T20:08:13.480 INFO:tasks.cephadm:Stopping all daemons... 2023-10-01T20:08:13.480 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-01T20:08:13.480 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.a 2023-10-01T20:08:13.761 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:13 smithi057 ceph-mon[209060]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 958 B/s rd, 0 op/s 2023-10-01T20:08:13.762 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:13 smithi057 ceph-mon[205388]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 958 B/s rd, 0 op/s 2023-10-01T20:08:13.762 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:13 smithi057 systemd[1]: Stopping Ceph mon.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:08:13.762 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-a[205365]: 2023-10-01T20:08:13.759+0000 7f3e48b28700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:08:13.763 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:13 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-a[205365]: 2023-10-01T20:08:13.759+0000 7f3e48b28700 -1 mon.a@0(leader) e4 *** Got Signal Terminated *** 2023-10-01T20:08:13.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:13 smithi087 ceph-mon[178735]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 958 B/s rd, 0 op/s 2023-10-01T20:08:14.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 01 20:08:14 smithi057 bash[222922]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-a 2023-10-01T20:08:14.354 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.a.service' 2023-10-01T20:08:14.416 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:08:14.417 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-01T20:08:14.417 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2023-10-01T20:08:14.417 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.c 2023-10-01T20:08:14.668 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:14 smithi057 systemd[1]: Stopping Ceph mon.c for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:08:14.922 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:08:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: ::ffff:172.21.15.87 - - [01/Oct/2023:20:08:14] "GET /metrics HTTP/1.1" 200 38001 "" "Prometheus/2.43.0" 2023-10-01T20:08:14.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-c[209036]: 2023-10-01T20:08:14.667+0000 7f24d319b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:08:14.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:14 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-c[209036]: 2023-10-01T20:08:14.667+0000 7f24d319b700 -1 mon.c@1(peon) e4 *** Got Signal Terminated *** 2023-10-01T20:08:14.922 INFO:journalctl@ceph.mon.c.smithi057.stdout:Oct 01 20:08:14 smithi057 bash[223067]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-c 2023-10-01T20:08:15.111 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.c.service' 2023-10-01T20:08:15.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:08:15.152 INFO:tasks.cephadm.mon.b:Stopped mon.c 2023-10-01T20:08:15.152 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2023-10-01T20:08:15.153 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.b 2023-10-01T20:08:15.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:15 smithi087 systemd[1]: Stopping Ceph mon.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:08:15.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:15 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-b[178710]: 2023-10-01T20:08:15.388+0000 7f79302ac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:08:15.620 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:15 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-b[178710]: 2023-10-01T20:08:15.388+0000 7f79302ac700 -1 mon.b@2(peon) e4 *** Got Signal Terminated *** 2023-10-01T20:08:15.883 INFO:journalctl@ceph.mon.b.smithi087.stdout:Oct 01 20:08:15 smithi087 bash[191470]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mon-b 2023-10-01T20:08:15.951 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mon.b.service' 2023-10-01T20:08:16.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:08:16.002 INFO:tasks.cephadm.mon.b:Stopped mon.b 2023-10-01T20:08:16.002 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-01T20:08:16.002 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.y 2023-10-01T20:08:16.285 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:08:16 smithi057 systemd[1]: Stopping Ceph mgr.y for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:08:16.285 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:08:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:08:16.249+0000 7fa4fe669700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:08:16.286 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:08:16 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y[161923]: 2023-10-01T20:08:16.249+0000 7fa4fe669700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-10-01T20:08:16.671 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Oct 01 20:08:16 smithi057 bash[223211]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-y 2023-10-01T20:08:16.787 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.y.service' 2023-10-01T20:08:16.828 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:08:16.828 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-01T20:08:16.829 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-01T20:08:16.829 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.x 2023-10-01T20:08:17.133 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:08:16 smithi087 systemd[1]: Stopping Ceph mgr.x for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:08:17.633 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:08:17 smithi087 bash[191648]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-mgr-x 2023-10-01T20:08:17.886 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.x.service' 2023-10-01T20:08:17.911 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:08:17 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.x.service: Main process exited, code=exited, status=143/n/a 2023-10-01T20:08:17.911 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:08:17 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@mgr.x.service: Failed with result 'exit-code'. 2023-10-01T20:08:17.911 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Oct 01 20:08:17 smithi087 systemd[1]: Stopped Ceph mgr.x for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:08:17.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:08:17.924 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-01T20:08:17.924 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-01T20:08:17.924 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.0 2023-10-01T20:08:18.421 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:08:17 smithi057 systemd[1]: Stopping Ceph osd.0 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:08:18.422 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:08:18 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[218943]: 2023-10-01T20:08:18.174+0000 7fe833df6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:08:18.422 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:08:18 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[218943]: 2023-10-01T20:08:18.174+0000 7fe833df6700 -1 osd.0 118 *** Got signal Terminated *** 2023-10-01T20:08:18.422 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:08:18 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0[218943]: 2023-10-01T20:08:18.174+0000 7fe833df6700 -1 osd.0 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-01T20:08:18.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:18 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:18.662Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:18.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:18 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:18.663Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:08:24.172 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 01 20:08:23 smithi057 bash[223356]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-0 2023-10-01T20:08:25.724 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.0.service' 2023-10-01T20:08:25.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:08:25.777 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-01T20:08:25.778 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-01T20:08:25.778 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.1 2023-10-01T20:08:26.171 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 20:08:25 smithi057 systemd[1]: Stopping Ceph osd.1 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:08:26.172 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 20:08:25 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-1[121553]: 2023-10-01T20:08:25.889+0000 7f61675ff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:08:26.172 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 20:08:25 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-1[121553]: 2023-10-01T20:08:25.889+0000 7f61675ff700 -1 osd.1 118 *** Got signal Terminated *** 2023-10-01T20:08:26.172 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 20:08:25 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-1[121553]: 2023-10-01T20:08:25.889+0000 7f61675ff700 -1 osd.1 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-01T20:08:28.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:28 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:28.662Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:28.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:28 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:28.663Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:28.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:28 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:28.663Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:08:31.672 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 20:08:31 smithi057 bash[224419]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-1 2023-10-01T20:08:31.672 INFO:journalctl@ceph.osd.1.smithi057.stdout:Oct 01 20:08:31 smithi057 bash[224479]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd.1" found: no such container 2023-10-01T20:08:31.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:31 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:31.633Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:31.672 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:31 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:31.634Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:08:32.563 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.1.service' 2023-10-01T20:08:32.617 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:08:32.617 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-01T20:08:32.617 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-01T20:08:32.618 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.2 2023-10-01T20:08:32.922 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 20:08:32 smithi057 systemd[1]: Stopping Ceph osd.2 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:08:32.922 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 20:08:32 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-2[124933]: 2023-10-01T20:08:32.731+0000 7fafe3859700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:08:32.922 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 20:08:32 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-2[124933]: 2023-10-01T20:08:32.731+0000 7fafe3859700 -1 osd.2 118 *** Got signal Terminated *** 2023-10-01T20:08:32.922 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 20:08:32 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-2[124933]: 2023-10-01T20:08:32.731+0000 7fafe3859700 -1 osd.2 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-01T20:08:38.663 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 20:08:38 smithi057 bash[224654]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-2 2023-10-01T20:08:38.664 INFO:journalctl@ceph.osd.2.smithi057.stdout:Oct 01 20:08:38 smithi057 bash[224697]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd.2" found: no such container 2023-10-01T20:08:38.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:38 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:38.663Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:08:38.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:38 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:38.664Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:38.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:38 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:38.664Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:08:39.503 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.2.service' 2023-10-01T20:08:39.556 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:08:39.556 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-01T20:08:39.556 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-01T20:08:39.556 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.3 2023-10-01T20:08:39.922 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 01 20:08:39 smithi057 systemd[1]: Stopping Ceph osd.3 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:08:39.922 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 01 20:08:39 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-3[128379]: 2023-10-01T20:08:39.654+0000 7f3ec1ee7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:08:39.922 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 01 20:08:39 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-3[128379]: 2023-10-01T20:08:39.654+0000 7f3ec1ee7700 -1 osd.3 118 *** Got signal Terminated *** 2023-10-01T20:08:39.922 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 01 20:08:39 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-3[128379]: 2023-10-01T20:08:39.654+0000 7f3ec1ee7700 -1 osd.3 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-01T20:08:41.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:41.633Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:41.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:41.634Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:41.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:41 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:41.634Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:08:44.134 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:43 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T20:08:43.766+0000 7fa3b7954700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:20.158506+0000 front 2023-10-01T20:08:20.158517+0000 (oldest deadline 2023-10-01T20:08:43.658424+0000) 2023-10-01T20:08:45.079 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:44 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T20:08:44.745+0000 7fa3b7954700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:20.158506+0000 front 2023-10-01T20:08:20.158517+0000 (oldest deadline 2023-10-01T20:08:43.658424+0000) 2023-10-01T20:08:45.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:45 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:45.078+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:45.422 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 01 20:08:45 smithi057 bash[224875]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-3 2023-10-01T20:08:45.422 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 01 20:08:45 smithi057 bash[225202]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd.3" found: no such container 2023-10-01T20:08:45.733 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:45 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:45.447+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:46.080 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:45 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T20:08:45.731+0000 7fa3b7954700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:20.158506+0000 front 2023-10-01T20:08:20.158517+0000 (oldest deadline 2023-10-01T20:08:43.658424+0000) 2023-10-01T20:08:46.271 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.3.service' 2023-10-01T20:08:46.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:08:46.321 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-01T20:08:46.321 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-01T20:08:46.321 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.4 2023-10-01T20:08:46.357 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:46 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:46.080+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:46.633 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:46 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:46.440+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:46.633 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:46 smithi087 systemd[1]: Stopping Ceph osd.4 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:08:46.634 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:46 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T20:08:46.443+0000 7fa3b9fb7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:08:46.634 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:46 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T20:08:46.443+0000 7fa3b9fb7700 -1 osd.4 118 *** Got signal Terminated *** 2023-10-01T20:08:46.634 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:46 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T20:08:46.443+0000 7fa3b9fb7700 -1 osd.4 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-01T20:08:47.048 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:46 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T20:08:46.765+0000 7fa3b7954700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:20.158506+0000 front 2023-10-01T20:08:20.158517+0000 (oldest deadline 2023-10-01T20:08:43.658424+0000) 2023-10-01T20:08:47.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:47 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:47.047+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:47.739 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:47 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:47.436+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:48.010 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:47 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T20:08:47.739+0000 7fa3b7954700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:20.158506+0000 front 2023-10-01T20:08:20.158517+0000 (oldest deadline 2023-10-01T20:08:43.658424+0000) 2023-10-01T20:08:48.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:48 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:48.009+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:48.766 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:48 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:48.448+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:48.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:48 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:48.664Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:48.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:48 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:48.664Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:48.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:48 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:48.665Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:08:49.133 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:48 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T20:08:48.765+0000 7fa3b7954700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:20.158506+0000 front 2023-10-01T20:08:20.158517+0000 (oldest deadline 2023-10-01T20:08:43.658424+0000) 2023-10-01T20:08:49.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:48 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:48.973+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:49.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:48 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:48.907+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:08:49.811 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:49 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:49.481+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:50.133 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:49 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T20:08:49.810+0000 7fa3b7954700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:20.158506+0000 front 2023-10-01T20:08:20.158517+0000 (oldest deadline 2023-10-01T20:08:43.658424+0000) 2023-10-01T20:08:50.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:49 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:49.964+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:50.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:49 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:49.886+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:08:50.858 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:50.439+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:51.133 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4[114801]: 2023-10-01T20:08:50.856+0000 7fa3b7954700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:20.158506+0000 front 2023-10-01T20:08:20.158517+0000 (oldest deadline 2023-10-01T20:08:43.658424+0000) 2023-10-01T20:08:51.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:50.969+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:51.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:50 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:50.918+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:08:51.883 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:51 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:51.444+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:51.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:51 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:51.634Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:51.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:51 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:51.634Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:51.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:51 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:51.635Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:08:52.383 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:52 smithi087 bash[192732]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-4 2023-10-01T20:08:52.383 INFO:journalctl@ceph.osd.4.smithi087.stdout:Oct 01 20:08:52 smithi087 bash[192775]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd.4" found: no such container 2023-10-01T20:08:52.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:51 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:51.980+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:52.384 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:51 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:51.916+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:08:52.384 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:51 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:51.916+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:08:52.866 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:52 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:52.477+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:52.867 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:52 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:52.477+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:26.484626+0000 front 2023-10-01T20:08:26.484720+0000 (oldest deadline 2023-10-01T20:08:52.384266+0000) 2023-10-01T20:08:53.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:53.019+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:53.134 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:53.019+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:08:53.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:52 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:52.901+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:08:53.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:52 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:52.901+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:08:53.189 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.4.service' 2023-10-01T20:08:53.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:08:53.271 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-01T20:08:53.271 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-01T20:08:53.272 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.5 2023-10-01T20:08:53.501 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:53 smithi087 systemd[1]: Stopping Ceph osd.5 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:08:53.502 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:53.400+0000 7f9fce1d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:08:53.502 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:53.400+0000 7f9fce1d4700 -1 osd.5 118 *** Got signal Terminated *** 2023-10-01T20:08:53.502 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:53.400+0000 7f9fce1d4700 -1 osd.5 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-01T20:08:53.883 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:53.501+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:53.883 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:53.501+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:26.484626+0000 front 2023-10-01T20:08:26.484720+0000 (oldest deadline 2023-10-01T20:08:52.384266+0000) 2023-10-01T20:08:54.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:54.007+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:54.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:54.007+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:08:54.384 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:53.922+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:08:54.384 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:53 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:53.922+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:08:54.883 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:54.501+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:54.883 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:54.501+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:26.484626+0000 front 2023-10-01T20:08:26.484720+0000 (oldest deadline 2023-10-01T20:08:52.384266+0000) 2023-10-01T20:08:55.383 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:54.898+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:08:55.383 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:54 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:54.898+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:08:55.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:55 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:55.008+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:55.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:55 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:55.008+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:08:55.883 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:55 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:55.454+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:55.883 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:55 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:55.454+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:26.484626+0000 front 2023-10-01T20:08:26.484720+0000 (oldest deadline 2023-10-01T20:08:52.384266+0000) 2023-10-01T20:08:56.180 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:56.015+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:56.181 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:56.015+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:08:56.181 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:55 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:55.914+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:08:56.181 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:55 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:55.914+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:08:56.883 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:56.498+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:56.883 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:56.498+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:26.484626+0000 front 2023-10-01T20:08:26.484720+0000 (oldest deadline 2023-10-01T20:08:52.384266+0000) 2023-10-01T20:08:57.226 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:57 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:57.021+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:57.226 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:57 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:57.022+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:08:57.226 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:56.959+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:08:57.226 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:56 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:56.959+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:08:57.633 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:57 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:57.449+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.483292+0000 front 2023-10-01T20:08:22.483276+0000 (oldest deadline 2023-10-01T20:08:45.383275+0000) 2023-10-01T20:08:57.633 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:57 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5[117992]: 2023-10-01T20:08:57.449+0000 7f9fc836a700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:26.484626+0000 front 2023-10-01T20:08:26.484720+0000 (oldest deadline 2023-10-01T20:08:52.384266+0000) 2023-10-01T20:08:58.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:58.053+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:58.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:58.053+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:08:58.384 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:57 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:57.915+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:08:58.384 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:57 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:57.915+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:08:58.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:58 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:58.664Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:58.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:58 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:58.665Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:08:58.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:08:58 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:08:58.665Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:08:59.089 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:59.011+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:08:59.089 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:59.011+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:08:59.089 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:08:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:08:59.011+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.061031+0000 front 2023-10-01T20:08:32.061061+0000 (oldest deadline 2023-10-01T20:08:58.461279+0000) 2023-10-01T20:08:59.089 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:58.895+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:08:59.089 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:58.895+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:08:59.090 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:58 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:58.895+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:08:59.383 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:59 smithi087 bash[192953]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-5 2023-10-01T20:08:59.383 INFO:journalctl@ceph.osd.5.smithi087.stdout:Oct 01 20:08:59 smithi087 bash[193668]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd.5" found: no such container 2023-10-01T20:09:00.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:00.004+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:09:00.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:00.004+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:09:00.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:00.004+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.061031+0000 front 2023-10-01T20:08:32.061061+0000 (oldest deadline 2023-10-01T20:08:58.461279+0000) 2023-10-01T20:09:00.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:59.859+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:00.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:59.859+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:00.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:08:59 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:08:59.859+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:00.805 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.5.service' 2023-10-01T20:09:00.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:09:00.848 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-01T20:09:00.848 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-01T20:09:00.848 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.6 2023-10-01T20:09:01.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:00.832+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:01.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:00.832+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:01.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:00.832+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:01.134 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:00 smithi087 systemd[1]: Stopping Ceph osd.6 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:09:01.134 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:00.983+0000 7f4133641700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:09:01.134 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:00.983+0000 7f4133641700 -1 osd.6 118 *** Got signal Terminated *** 2023-10-01T20:09:01.134 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:00 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:00.983+0000 7f4133641700 -1 osd.6 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-01T20:09:01.134 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:01 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:01.006+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:09:01.135 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:01 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:01.006+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:09:01.135 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:01 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:01.006+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.061031+0000 front 2023-10-01T20:08:32.061061+0000 (oldest deadline 2023-10-01T20:08:58.461279+0000) 2023-10-01T20:09:01.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:01 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:09:01.635Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:09:01.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:01 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:09:01.635Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:09:01.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:01 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:09:01.636Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:09:02.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:02.052+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:09:02.134 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:02.052+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:09:02.134 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:02.052+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.061031+0000 front 2023-10-01T20:08:32.061061+0000 (oldest deadline 2023-10-01T20:08:58.461279+0000) 2023-10-01T20:09:02.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:01 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:01.838+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:02.135 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:01 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:01.838+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:02.135 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:01 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:01.838+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:03.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:03.052+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:09:03.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:03.052+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:09:03.134 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:03.052+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.061031+0000 front 2023-10-01T20:08:32.061061+0000 (oldest deadline 2023-10-01T20:08:58.461279+0000) 2023-10-01T20:09:03.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:02.803+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:03.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:02.803+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:03.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:02 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:02.803+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:04.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:04.033+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:09:04.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:04.033+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:09:04.133 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:04.033+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.061031+0000 front 2023-10-01T20:08:32.061061+0000 (oldest deadline 2023-10-01T20:08:58.461279+0000) 2023-10-01T20:09:04.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:03.792+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:04.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:03.792+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:04.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:03 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:03.792+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:05.057 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:04.750+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:05.057 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:04.750+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:05.058 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:04 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:04.750+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:05.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:05.056+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.859228+0000 front 2023-10-01T20:08:22.859019+0000 (oldest deadline 2023-10-01T20:08:44.559166+0000) 2023-10-01T20:09:05.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:05.056+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.360730+0000 front 2023-10-01T20:08:27.360827+0000 (oldest deadline 2023-10-01T20:08:52.060554+0000) 2023-10-01T20:09:05.383 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6[121357]: 2023-10-01T20:09:05.056+0000 7f412e7d9700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.061031+0000 front 2023-10-01T20:08:32.061061+0000 (oldest deadline 2023-10-01T20:08:58.461279+0000) 2023-10-01T20:09:05.383 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:05.151Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.57:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.57:8765: connect: connection refused" 2023-10-01T20:09:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:05.151Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.57:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.57:8765: connect: connection refused" 2023-10-01T20:09:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:05.154Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.57:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.57:8765: connect: connection refused" 2023-10-01T20:09:05.384 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:05.154Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.57:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.57:8765: connect: connection refused" 2023-10-01T20:09:06.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:05.729+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:06.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:05.729+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:06.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:05 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:05.729+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:06.723 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:06 smithi087 bash[194013]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-6 2023-10-01T20:09:06.724 INFO:journalctl@ceph.osd.6.smithi087.stdout:Oct 01 20:09:06 smithi087 bash[194056]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd.6" found: no such container 2023-10-01T20:09:07.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:06 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:06.722+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:07.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:06 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:06.722+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:07.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:06 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:06.722+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:07.522 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.6.service' 2023-10-01T20:09:07.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:09:07.564 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-01T20:09:07.565 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-01T20:09:07.565 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.7 2023-10-01T20:09:07.883 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:07 smithi087 systemd[1]: Stopping Ceph osd.7 for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:09:07.883 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:07 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:07.699+0000 7fe0d28a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-01T20:09:07.883 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:07 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:07.699+0000 7fe0d28a1700 -1 osd.7 118 *** Got signal Terminated *** 2023-10-01T20:09:07.884 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:07 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:07.699+0000 7fe0d28a1700 -1 osd.7 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-01T20:09:07.884 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:07 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:07.701+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:07.884 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:07 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:07.701+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:07.884 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:07 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:07.701+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:08.925 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:08 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:09:08.665Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:09:08.926 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:08 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:09:08.666Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:09:08.926 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:08 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:09:08.666Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:09:09.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:08 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:08.677+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:09.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:08 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:08.677+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:09.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:08 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:08.677+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:10.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:09 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:09.648+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:10.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:09 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:09.648+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:10.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:09 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:09.648+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:10.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:09 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:09.648+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6830 osd.3 since back 2023-10-01T20:08:43.645868+0000 front 2023-10-01T20:08:43.645857+0000 (oldest deadline 2023-10-01T20:09:08.945872+0000) 2023-10-01T20:09:11.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:10.660+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:11.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:10.660+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:11.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:10.660+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:11.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:10 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:10.660+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6830 osd.3 since back 2023-10-01T20:08:43.645868+0000 front 2023-10-01T20:08:43.645857+0000 (oldest deadline 2023-10-01T20:09:08.945872+0000) 2023-10-01T20:09:11.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:09:11.636Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:09:11.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:09:11.637Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi057.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.57:8443: connect: connection refused" 2023-10-01T20:09:11.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:11 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:09:11.637Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2023-10-01T20:09:12.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:11 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:11.676+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:12.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:11 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:11.676+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:12.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:11 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:11.676+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:12.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:11 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:11.676+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6830 osd.3 since back 2023-10-01T20:08:43.645868+0000 front 2023-10-01T20:08:43.645857+0000 (oldest deadline 2023-10-01T20:09:08.945872+0000) 2023-10-01T20:09:13.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:12 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:12.661+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-01T20:08:22.043089+0000 front 2023-10-01T20:08:22.043077+0000 (oldest deadline 2023-10-01T20:08:47.942971+0000) 2023-10-01T20:09:13.133 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:12 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:12.661+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2023-10-01T20:08:27.943452+0000 front 2023-10-01T20:08:27.943614+0000 (oldest deadline 2023-10-01T20:08:51.443370+0000) 2023-10-01T20:09:13.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:12 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:12.661+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2023-10-01T20:08:32.544586+0000 front 2023-10-01T20:08:32.544695+0000 (oldest deadline 2023-10-01T20:08:58.444467+0000) 2023-10-01T20:09:13.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:12 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:12.661+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6830 osd.3 since back 2023-10-01T20:08:43.645868+0000 front 2023-10-01T20:08:43.645857+0000 (oldest deadline 2023-10-01T20:09:08.945872+0000) 2023-10-01T20:09:13.134 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:12 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7[124729]: 2023-10-01T20:09:12.662+0000 7fe0cba35700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.87:6804 osd.4 since back 2023-10-01T20:08:49.446657+0000 front 2023-10-01T20:08:49.446738+0000 (oldest deadline 2023-10-01T20:09:12.247343+0000) 2023-10-01T20:09:13.554 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:13 smithi087 bash[194231]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd-7 2023-10-01T20:09:13.554 INFO:journalctl@ceph.osd.7.smithi087.stdout:Oct 01 20:09:13 smithi087 bash[194273]: Error: no container with name or ID "ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-osd.7" found: no such container 2023-10-01T20:09:14.280 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@osd.7.service' 2023-10-01T20:09:14.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:09:14.321 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-01T20:09:14.321 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2023-10-01T20:09:14.321 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@prometheus.a 2023-10-01T20:09:14.583 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 systemd[1]: Stopping Ceph prometheus.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:09:14.584 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:14.581Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2023-10-01T20:09:14.584 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:14.581Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2023-10-01T20:09:14.584 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:14.581Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2023-10-01T20:09:14.584 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:14.581Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2023-10-01T20:09:14.584 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:14.581Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2023-10-01T20:09:14.584 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:14.581Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2023-10-01T20:09:14.584 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:14.582Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2023-10-01T20:09:14.584 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:14.582Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2023-10-01T20:09:14.585 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:14.582Z caller=main.go:907 level=info msg="Scrape manager stopped" 2023-10-01T20:09:14.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:14.585Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2023-10-01T20:09:14.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:14.585Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2023-10-01T20:09:14.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a[169100]: ts=2023-10-01T20:09:14.585Z caller=main.go:1147 level=info msg="See you next time!" 2023-10-01T20:09:14.883 INFO:journalctl@ceph.prometheus.a.smithi087.stdout:Oct 01 20:09:14 smithi087 bash[194476]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-prometheus-a 2023-10-01T20:09:15.059 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@prometheus.a.service' 2023-10-01T20:09:15.095 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-01T20:09:15.095 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2023-10-01T20:09:15.096 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-01T20:09:16.422 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:09:16 smithi057 systemd[1]: Stopping Ceph node-exporter.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:09:17.161 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:09:16 smithi057 bash[226559]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-a 2023-10-01T20:09:17.161 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:09:16 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2023-10-01T20:09:17.422 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:09:17 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.a.service: Failed with result 'exit-code'. 2023-10-01T20:09:17.422 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Oct 01 20:09:17 smithi057 systemd[1]: Stopped Ceph node-exporter.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:09:17.422 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:17 smithi057 systemd[1]: Stopping Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:09:17.921 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:17 smithi057 ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a[190331]: ts=2023-10-01T20:09:17.499Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2023-10-01T20:09:17.922 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:17 smithi057 bash[226688]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-alertmanager-a 2023-10-01T20:09:18.422 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:18 smithi057 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@alertmanager.a.service: Succeeded. 2023-10-01T20:09:18.422 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Oct 01 20:09:18 smithi057 systemd[1]: Stopped Ceph alertmanager.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:09:39.657 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-01T20:09:40.883 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:09:40 smithi087 systemd[1]: Stopping Ceph node-exporter.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:09:41.383 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:09:41 smithi087 bash[195994]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-node-exporter-b 2023-10-01T20:09:41.383 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:09:41 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2023-10-01T20:09:41.808 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:09:41 smithi087 systemd[1]: Stopping Ceph grafana.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7... 2023-10-01T20:09:41.809 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:09:41 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@node-exporter.b.service: Failed with result 'exit-code'. 2023-10-01T20:09:41.809 INFO:journalctl@ceph.node-exporter.b.smithi087.stdout:Oct 01 20:09:41 smithi087 systemd[1]: Stopped Ceph node-exporter.b for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:09:42.133 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:09:41 smithi087 bash[196123]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7-grafana-a 2023-10-01T20:09:42.133 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:09:41 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@grafana.a.service: Main process exited, code=exited, status=143/n/a 2023-10-01T20:09:42.633 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:09:42 smithi087 systemd[1]: ceph-0f6d5dc0-6092-11ee-8db5-212e2dc638e7@grafana.a.service: Failed with result 'exit-code'. 2023-10-01T20:09:42.634 INFO:journalctl@ceph.grafana.a.smithi087.stdout:Oct 01 20:09:42 smithi087 systemd[1]: Stopped Ceph grafana.a for 0f6d5dc0-6092-11ee-8db5-212e2dc638e7. 2023-10-01T20:09:52.919 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-01T20:09:52.962 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-01T20:09:52.994 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-01T20:09:52.996 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414/remote/smithi057/crash 2023-10-01T20:09:52.996 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/crash -- . 2023-10-01T20:09:53.049 INFO:teuthology.orchestra.run.smithi057.stderr:tar: /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-01T20:09:53.049 INFO:teuthology.orchestra.run.smithi057.stderr:tar: Error is not recoverable: exiting now 2023-10-01T20:09:53.051 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414/remote/smithi087/crash 2023-10-01T20:09:53.052 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/crash -- . 2023-10-01T20:09:53.088 INFO:teuthology.orchestra.run.smithi087.stderr:tar: /var/lib/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-01T20:09:53.088 INFO:teuthology.orchestra.run.smithi087.stderr:tar: Error is not recoverable: exiting now 2023-10-01T20:09:53.089 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-01T20:09:53.089 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2023-10-01T20:09:53.124 INFO:teuthology.orchestra.run.smithi057.stderr:grep: /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph.log: No such file or directory 2023-10-01T20:09:53.127 INFO:tasks.cephadm:Compressing logs... 2023-10-01T20:09:53.127 DEBUG:teuthology.orchestra.run.smithi057:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-01T20:09:53.170 DEBUG:teuthology.orchestra.run.smithi087:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-01T20:09:53.198 INFO:teuthology.orchestra.run.smithi087.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose : No such file or directory-- /var/log/ceph/cephadm.log 2023-10-01T20:09:53.198 INFO:teuthology.orchestra.run.smithi087.stderr: 2023-10-01T20:09:53.198 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.6.log 2023-10-01T20:09:53.198 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose /var/log/ceph/cephadm.log: -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-01T20:09:53.198 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-client.rgw.foo.smithi087.rxeurp.log 2023-10-01T20:09:53.200 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.7.log 2023-10-01T20:09:53.200 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-client.rgw.foo.smithi087.rxeurp.log: gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mgr.x.log 2023-10-01T20:09:53.200 INFO:teuthology.orchestra.run.smithi087.stderr: 62.7% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-client.rgw.foo.smithi087.rxeurp.log.gz 2023-10-01T20:09:53.200 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mon.b.log 2023-10-01T20:09:53.200 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.4.log 2023-10-01T20:09:53.202 INFO:teuthology.orchestra.run.smithi057.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-01T20:09:53.202 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-01T20:09:53.203 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-01T20:09:53.203 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.2.log 2023-10-01T20:09:53.204 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mon.b.log: /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.5.log 2023-10-01T20:09:53.205 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mgr.y.log 2023-10-01T20:09:53.205 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.3.log 2023-10-01T20:09:53.205 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.2.log: /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.0.log 2023-10-01T20:09:53.211 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.3.log: /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mon.a.log 2023-10-01T20:09:53.212 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mon.c.log 2023-10-01T20:09:53.216 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mon.a.log: /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.1.log 2023-10-01T20:09:53.226 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.5.log: 88.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-01T20:09:53.239 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.1.log: 90.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-01T20:09:53.239 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/tcmu-runner.log 2023-10-01T20:09:53.239 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/tcmu-runner.log: 86.2% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/tcmu-runner.log.gz 2023-10-01T20:09:53.239 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-client.rgw.foo.smithi057.dbidoa.log 2023-10-01T20:09:53.239 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-client.rgw.foo.smithi057.dbidoa.log: 62.6% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-client.rgw.foo.smithi057.dbidoa.log.gz 2023-10-01T20:09:53.251 INFO:teuthology.orchestra.run.smithi087.stderr: 92.3% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-01T20:09:53.259 INFO:teuthology.orchestra.run.smithi057.stderr: 92.3% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-01T20:09:53.364 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mgr.x.log.gz 2023-10-01T20:09:53.744 INFO:teuthology.orchestra.run.smithi087.stderr: 92.6% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mon.b.log.gz 2023-10-01T20:09:53.828 INFO:teuthology.orchestra.run.smithi057.stderr: 92.6% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mon.c.log.gz 2023-10-01T20:09:53.893 INFO:teuthology.orchestra.run.smithi057.stderr: 88.0% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mgr.y.log.gz 2023-10-01T20:09:54.266 INFO:teuthology.orchestra.run.smithi087.stderr: 93.0% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.6.log.gz 2023-10-01T20:09:54.321 INFO:teuthology.orchestra.run.smithi087.stderr: 93.8% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.7.log.gz 2023-10-01T20:09:54.384 INFO:teuthology.orchestra.run.smithi057.stderr: 93.2% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.2.log.gz 2023-10-01T20:09:54.389 INFO:teuthology.orchestra.run.smithi087.stderr: 93.2% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.5.log.gz 2023-10-01T20:09:54.581 INFO:teuthology.orchestra.run.smithi057.stderr: 91.5% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-mon.a.log.gz 2023-10-01T20:09:54.608 INFO:teuthology.orchestra.run.smithi087.stderr: 93.4% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.4.log.gz 2023-10-01T20:09:54.610 INFO:teuthology.orchestra.run.smithi087.stderr: 2023-10-01T20:09:54.610 INFO:teuthology.orchestra.run.smithi087.stderr:real 0m1.423s 2023-10-01T20:09:54.610 INFO:teuthology.orchestra.run.smithi087.stderr:user 0m5.358s 2023-10-01T20:09:54.610 INFO:teuthology.orchestra.run.smithi087.stderr:sys 0m0.233s 2023-10-01T20:09:54.618 INFO:teuthology.orchestra.run.smithi057.stderr: 93.4% 93.4% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.0.log.gz -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.1.log.gz 2023-10-01T20:09:54.618 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-01T20:09:55.071 INFO:teuthology.orchestra.run.smithi057.stderr: 93.3% -- replaced with /var/log/ceph/0f6d5dc0-6092-11ee-8db5-212e2dc638e7/ceph-osd.3.log.gz 2023-10-01T20:09:55.073 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-01T20:09:55.073 INFO:teuthology.orchestra.run.smithi057.stderr:real 0m1.883s 2023-10-01T20:09:55.073 INFO:teuthology.orchestra.run.smithi057.stderr:user 0m8.263s 2023-10-01T20:09:55.073 INFO:teuthology.orchestra.run.smithi057.stderr:sys 0m0.352s 2023-10-01T20:09:55.074 INFO:tasks.cephadm:Archiving logs... 2023-10-01T20:09:55.074 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/log/ceph to /home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414/remote/smithi057/log 2023-10-01T20:09:55.075 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-01T20:09:56.206 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/log/ceph to /home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414/remote/smithi087/log 2023-10-01T20:09:56.207 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-01T20:09:57.007 INFO:tasks.cephadm:Removing cluster... 2023-10-01T20:09:57.007 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 --force 2023-10-01T20:09:57.360 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 --force 2023-10-01T20:09:57.689 INFO:tasks.cephadm:Removing cephadm ... 2023-10-01T20:09:57.689 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-01T20:09:57.713 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-01T20:09:57.731 INFO:tasks.cephadm:Teardown complete 2023-10-01T20:09:57.732 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-01T20:09:57.742 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-01T20:09:57.743 DEBUG:teuthology.orchestra.run.smithi057:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-01T20:09:57.755 DEBUG:teuthology.orchestra.run.smithi087:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-01T20:09:57.775 INFO:teuthology.orchestra.run.smithi057.stderr:bash: ntpq: command not found 2023-10-01T20:09:57.781 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-01T20:09:57.781 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2023-10-01T20:09:57.781 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 2 -774us[ -774us] +/- 97ms 2023-10-01T20:09:57.781 INFO:teuthology.orchestra.run.smithi057.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 -384us[ -393us] +/- 67ms 2023-10-01T20:09:57.781 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +859us[ +859us] +/- 65ms 2023-10-01T20:09:57.781 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-01T20:09:57.788 INFO:teuthology.orchestra.run.smithi087.stderr:bash: ntpq: command not found 2023-10-01T20:09:57.793 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-01T20:09:57.793 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2023-10-01T20:09:57.793 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 1 -771us[ -771us] +/- 97ms 2023-10-01T20:09:57.793 INFO:teuthology.orchestra.run.smithi087.stdout:^* hv02.front.sepia.ceph.com 3 7 377 4 -362us[ -373us] +/- 67ms 2023-10-01T20:09:57.793 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 +854us[ +843us] +/- 65ms 2023-10-01T20:09:57.794 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-01T20:09:57.794 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-01T20:09:57.803 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-01T20:09:57.803 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-01T20:09:57.812 DEBUG:teuthology.orchestra.run.smithi057:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-01T20:09:57.943 DEBUG:teuthology.orchestra.run.smithi087:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-01T20:09:58.044 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-01T20:09:58.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-01T20:09:58.085 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-01T20:09:58.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-01T20:09:58.118 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-01T20:09:58.127 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-01T20:09:58.135 INFO:teuthology.task.internal:Duration was 2501.444057 seconds 2023-10-01T20:09:58.136 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-01T20:09:58.145 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-01T20:09:58.145 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-01T20:09:58.148 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-01T20:09:58.203 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-01T20:09:58.205 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-01T20:09:58.684 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-01T20:09:58.684 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi057.front.sepia.ceph.com 2023-10-01T20:09:58.684 DEBUG:teuthology.orchestra.run.smithi057:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-01T20:09:58.714 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi087.front.sepia.ceph.com 2023-10-01T20:09:58.714 DEBUG:teuthology.orchestra.run.smithi087:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-01T20:09:58.738 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-01T20:09:58.738 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-01T20:09:58.756 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-01T20:09:58.844 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-01T20:09:58.845 DEBUG:teuthology.orchestra.run.smithi057:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-01T20:09:58.847 DEBUG:teuthology.orchestra.run.smithi087:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-01T20:09:59.068 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-01T20:09:59.079 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-01T20:09:59.079 DEBUG:teuthology.orchestra.run.smithi057:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-01T20:09:59.111 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-01T20:09:59.139 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-01T20:09:59.149 DEBUG:teuthology.orchestra.run.smithi057:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-01T20:09:59.153 DEBUG:teuthology.orchestra.run.smithi087:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-01T20:09:59.184 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = core 2023-10-01T20:09:59.207 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = core 2023-10-01T20:09:59.249 DEBUG:teuthology.orchestra.run.smithi057:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-01T20:09:59.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-01T20:09:59.273 DEBUG:teuthology.orchestra.run.smithi087:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-01T20:09:59.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-01T20:09:59.306 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-01T20:09:59.316 INFO:teuthology.task.internal:Transferring archived files... 2023-10-01T20:09:59.317 DEBUG:teuthology.misc:Transferring archived files from smithi057:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414/remote/smithi057 2023-10-01T20:09:59.317 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-01T20:09:59.374 DEBUG:teuthology.misc:Transferring archived files from smithi087:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414/remote/smithi087 2023-10-01T20:09:59.374 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-01T20:09:59.419 INFO:teuthology.task.internal:Removing archive directory... 2023-10-01T20:09:59.419 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-01T20:09:59.422 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-01T20:09:59.468 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-01T20:09:59.480 INFO:teuthology.task.internal:Not uploading archives. 2023-10-01T20:09:59.480 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-01T20:09:59.490 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-01T20:09:59.490 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-01T20:09:59.493 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-01T20:09:59.513 INFO:teuthology.orchestra.run.smithi057.stdout: 393225 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 1 20:09 /home/ubuntu/cephtest 2023-10-01T20:09:59.526 INFO:teuthology.orchestra.run.smithi087.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 1 20:09 /home/ubuntu/cephtest 2023-10-01T20:09:59.527 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-01T20:09:59.537 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-01T20:09:59.557 INFO:teuthology.nuke:Checking targets against current locks 2023-10-01T20:09:59.584 DEBUG:teuthology.nuke:shortname: smithi057 2023-10-01T20:09:59.584 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-01T20:09:59.608 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-01 19:16:11.702963', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK8YYlq2Rp06VHBEX27K/8AkZc8R3eU+RtuZM71chaqD+0nnh/8jTrdaEObMg5q0X09JtzTT68xaayDSm/pyU3A='} 2023-10-01T20:09:59.611 DEBUG:teuthology.nuke:shortname: smithi087 2023-10-01T20:09:59.612 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-01T20:09:59.634 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-10-01_19:14:56-orch:cephadm-wip-adk-testing-2023-10-01-1310-distro-default-smithi/7408414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-01 19:16:11.704601', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEzI4F+XBj70CitNOaqAwYigOp8SsukyIckXGTxr4S3m4S3WXm5I+CNA4etqkLkOk6UqNbV5z3YR8LD+RguoOH0='} 2023-10-01T20:09:59.668 INFO:teuthology.orchestra.console.smithi057:Power off 2023-10-01T20:09:59.668 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-01T20:09:59.693 DEBUG:teuthology.orchestra.console.smithi057:power off output: Chassis Power Control: Down/Off 2023-10-01T20:09:59.694 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-01T20:09:59.710 INFO:teuthology.orchestra.console.smithi087:Power off 2023-10-01T20:09:59.710 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-01T20:09:59.721 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-10-01T20:09:59.732 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off 2023-10-01T20:09:59.732 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-01T20:09:59.754 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2023-10-01T20:10:03.723 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-01T20:10:03.756 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-01T20:10:03.834 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-10-01T20:10:03.866 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2023-10-01T20:10:07.838 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-01T20:10:07.867 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-01T20:10:07.949 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-10-01T20:10:07.978 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-10-01T20:10:08.079 INFO:teuthology.orchestra.console.smithi087:Power off completed 2023-10-01T20:10:11.950 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-01T20:10:12.061 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is off 2023-10-01T20:10:12.162 INFO:teuthology.orchestra.console.smithi057:Power off completed 2023-10-01T20:10:12.266 INFO:teuthology.run:Summary data: description: orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} duration: 2501.4440565109253 failure_reason: 'Command failed on smithi057 with status 1: ''sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f6d5dc0-6092-11ee-8db5-212e2dc638e7 -e sha1=37d71fd39f09a164b3518d8aef9e20133197e5f9 -- bash -c \''ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \''"\''"\''.up_to_date | length == 7\''"\''"\''\''''' owner: scheduled_adking@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=933d3ec517e64d5599515ac11339f85d status: fail success: false 2023-10-01T20:10:12.267 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-01T20:10:12.352 INFO:teuthology.run:FAIL